This document is the summary of the Introduction to R workshop.
All correspondence related to this document should be addressed to:
Omid Ghasemi (Macquarie University, Sydney, NSW, 2109, AUSTRALIA)
Email: omidreza.ghasemi@hdr.mq.edu.au
Research Question
The aim of the study is to test if simple arguments are more effective in belief revision than more complex arguments. To that end, we present participants with an imaginary scenario (two alien creatures on a planet) and a theory (one creature is predator and the other one is prey) and we ask them to rate the likelihood truth of the theory based on a simple fact (We adapted this method from Gregg et al.,2017; see the original study here). Then, in a between-subject manipulation, participants will be presented with either 6 simple arguments (Modus Ponens conditionals) or 6 more complex arguments (Modus Tollens conditionals), and they will be asked to rate the likelihood truth of the initial theory on 7 stages.
The first stage is the base rating stage. The next three stages include supportive arguments of the theory and the last three arguments include disproving arguments of the theory. We hypothesized that the group with simple arguments shows better persuasion (as it reflects in higher ratings for the supportive arguments) and better dissuasion (as it reflects in lower ratings for the opposing arguments).
In the last part of the study, participants will be asked to answer several cognitive capacity/style measures including CRT, AOT-E, mindware, and numeracy scales. We hypothesized that cognitive ability, cognitive style, and open-mindedness are positive predictors of persuasion and dissuasion. These associations should be more pronounced for participants in the group with complex arguments because the ability and willingness to engage in deliberative thinking may favor participants to assess the underlying logical structure of those arguments. However, for participants in the simple group, the logical structure of arguments is more evident, so participants with lower ability can still assess the logical status of those arguments.

Thus, our hypotheses for this experiment are as follows:
Participants in the group with simple arguments have higher ratings for supportive arguments (They are more easily persuaded than those in the group with complex arguments).
Participants in the group with simple arguments have lower ratings for opposing arguments (They are more easily dissuaded than those in the group with complex arguments).
There are significant associations between CRT, AOT-E, Numeracy, and mindware with both persuasion and dissuasion indexes in each group and in the entire sample. The relationship between these measures should be stronger, although not significantly, for participants in the group with complex arguments.

Getting Ready
First, we need to design the experiment. For this experiment, we use online platforms for data collection. There are several options such as Gorilla, JSpsych, Qualtrics, psychoJS (pavlovia), etc. Since we do not need any reaction time data, we simply use Qualtrics. For an overview of different lab-based and online platforms, see here.
Next, we need to decide on the number of participants (sample size). For this study, we do not sue power analysis since we cannot access more than 120 participants. However, it is highly suggested calculate sample size using power estimation. You can find some nice tutorials on how to do that here, here, and here.
After we created the experiment and decided on the sample size, the next step is to preresigter the study. However, it would be better to do a pilot with 4 or 5 participants, clean all the data, do the desired analysis, and then pre-register the analysis and those codes. You can find the preregistration form for the current study here.
Finally, we need to restructure our project in a tidy folder with different sub-folders. Having a clean and tidy folder structure can save us! There are different formats of folder structure (for example, see here and here), but for now, we use the following structure:

Introduction to R
# load libraries
library(tidyverse)
library(here)
library(janitor)
library(broom)
library(afex)
library(emmeans)
library(knitr)
library(kableExtra)
library(ggsci)
library(patchwork)
library(skimr)
# install.packages("devtools")
# devtools::install_github("easystats/correlation")
library("correlation")
options(scipen=999) # turn off scientific notations
options(contrasts = c('contr.sum','contr.poly')) # set the contrast sum globally
options(knitr.kable.NA = '')
R can be used as a calculator. For mathematical purposes, be careful of the order in which R executes the commands.
10 + 10
## [1] 20
4 ^ 2
## [1] 16
(250 / 500) * 100
## [1] 50
R is a bit flexible with spacing (but no spacing in the name of variables and words)
10+10
## [1] 20
10 + 10
## [1] 20
R can sometimes tell that you’re not finished yet
10 +
How to create a variable? Variable assignment using <- and =. Note that R is case sensitive for everything
pay <- 250
month = 12
pay * month
## [1] 3000
salary <- pay * month
Few points in naming variables and vectors: use short, informative words, keep same method (e.g., not using capital words, use only _ or . ).
Function
Function is a set of statements combined together to perform a specific task. When we use a block of code repeatedly, we can convert it to a function. To write a function, first, you need to define it:
my_multiplier <- function(a,b){
result = a * b
return (result)
}
This code do nothing. To get a result, you need to call it:
my_multiplier (2,4)
## [1] 8
Fortunately, you do not need to write everything from scratch. R has lots of built-in functions that you can use:
round(54.6787)
## [1] 55
round(54.5787, digits = 2)
## [1] 54.58
Use ? before the function name to get some help. For example, ?round. You will see many functions in the rest of the workshop.
Basic Data Types in R:
function class() is used to show what is the type of a variable.
- Logical:
TRUE, FALSE can be abbreviated as T, F. They has to be capital, ‘true’ is not a logical data:
class(TRUE)
## [1] "logical"
class(F)
## [1] "logical"
- Numeric: all numbers e.g. 5, 10.5, 11,37; a special type of numeric is “integer” which is numbers without decimal. Integers are always numeric, but numeric is not always integer:
class(2)
## [1] "numeric"
class(13.46)
## [1] "numeric"
- Character: text for example, “I love R” or “4” or “4.5”:
class("ha ha ha ha")
## [1] "character"
class("56.6")
## [1] "character"
class("TRUE")
## [1] "character"
Can we change the type of data in a variable? Yes, you need to use the function as.---()
as.numeric(TRUE)
## [1] 1
as.character(4)
## [1] "4"
as.numeric("4.5")
## [1] 4.5
as.numeric("Hello")
## Warning: NAs introduced by coercion
## [1] NA
Data Structures in R
Vector: when there are more than one number or letter stored. Use the combine function c() for that.
sale <- c(1, 2, 3,4, 5, 6, 7, 8, 9, 10) # also sale <- c(1:10)
sale <- c(1:10)
sale * sale
## [1] 1 4 9 16 25 36 49 64 81 100
Subsetting a vector:
days <- c("Saturday", "Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday")
days[2]
## [1] "Sunday"
days[-2]
## [1] "Saturday" "Monday" "Tuesday" "Wednesday" "Thursday" "Friday"
days[c(2, 3, 4)]
## [1] "Sunday" "Monday" "Tuesday"
Exercise
Create a vector named my_vector with numbers from 0 to 1000 in it:
my_vector <- (0:1000)
mean(my_vector)
## [1] 500
median(my_vector)
## [1] 500
min(my_vector)
## [1] 0
range(my_vector)
## [1] 0 1000
class(my_vector)
## [1] "integer"
sum(my_vector)
## [1] 500500
sd(my_vector)
## [1] 289.1081
List: allows you to gather a variety of objects under one name (that is, the name of the list) in an ordered way. These objects can be matrices, vectors, data frames, even other list.
my_list = list(sale, 1, 3, 4:7, "HELLO", "hello", FALSE)
my_list
## [[1]]
## [1] 1 2 3 4 5 6 7 8 9 10
##
## [[2]]
## [1] 1
##
## [[3]]
## [1] 3
##
## [[4]]
## [1] 4 5 6 7
##
## [[5]]
## [1] "HELLO"
##
## [[6]]
## [1] "hello"
##
## [[7]]
## [1] FALSE
Factor: Factors store the vector along with the distinct values of the elements in the vector as labels. The labels are always character irrespective of whether it is numeric or character. For example, variable gender with “male” and “female” entries:
gender <- c("male", "male", "male", " female", "female", "female")
gender <- factor(gender)
R now treats gender as a nominal (categorical) variable: 1=female, 2=male internally (alphabetically).
summary(gender)
## female female male
## 1 2 3
Question: why when we ran the above function i.e. summary(), it showed three and not two levels of the data? Hint: run ‘gender’.
gender
## [1] male male male female female female
## Levels: female female male
So, be careful of spaces!
Exercise
Create a gender factor with 30 male and 40 females (Hint: use the rep() function):
gender <- c(rep("male",30), rep("female", 40))
gender <- factor(gender)
gender
## [1] male male male male male male male male male male
## [11] male male male male male male male male male male
## [21] male male male male male male male male male male
## [31] female female female female female female female female female female
## [41] female female female female female female female female female female
## [51] female female female female female female female female female female
## [61] female female female female female female female female female female
## Levels: female male
There are two types of categorical variables: nominal and ordinal. How to create ordered factors (when the variable is nominal and values can be ordered)? We should add two additional arguments to the factor() function: ordered = TRUE, and levels = c("level1", "level2"). For example, we have a vector that shows participants’ education level.
edu<-c(3,2,3,4,1,2,2,3,4)
education<-factor(edu, ordered = TRUE)
levels(education) <- c("Primary school","high school","College","Uni graduated")
education
## [1] College high school College Uni graduated
## [5] Primary school high school high school College
## [9] Uni graduated
## Levels: Primary school < high school < College < Uni graduated
Exercise
We have a factor with patient and control values. Here, the first level is control and the second level is patient. Change the order of levels, so patient would be the first level:
health_status <- factor(c(rep('patient',5),rep('control',5)))
health_status
## [1] patient patient patient patient patient control control control
## [9] control control
## Levels: control patient
health_status_reordered <- factor(health_status, levels = c('patient','control'))
health_status_reordered
## [1] patient patient patient patient patient control control control
## [9] control control
## Levels: patient control
Finally, can you relabel both levels to uppercase characters? (Hint: check ?factor)
health_status_relabeled <- factor(health_status, levels = c('patient','control'), labels = c('Patient','Control'))
health_status_relabeled
## [1] Patient Patient Patient Patient Patient Control Control Control
## [9] Control Control
## Levels: Patient Control
Matrices: All columns in a matrix must have the same mode(numeric, character, etc.) and the same length. It can be created using a vector input to the matrix function.
my_matrix = matrix(c(1,2,3,4,5,6,7,8,9), nrow = 3, ncol = 3)
my_matrix
## [,1] [,2] [,3]
## [1,] 1 4 7
## [2,] 2 5 8
## [3,] 3 6 9
Data frames: (two-dimensional objects) can hold numeric, character or logical values. Within a column all elements have the same data type, but different columns can be of different data type. Let’s create a dataframe:
id <- 1:200
group <- c(rep("Psychotherapy", 100), rep("Medication", 100))
response <- c(rnorm(100, mean = 30, sd = 5),
rnorm(100, mean = 25, sd = 5))
my_dataframe <-data.frame(Patient = id,
Treatment = group,
Response = response)
We also could have done the below
my_dataframe <-data.frame(Patient = c(1:200),
Treatment = c(rep("Psychotherapy", 100), rep("Medication", 100)),
Response = c(rnorm(100, mean = 30, sd = 5),
rnorm(100, mean = 25, sd = 5)))
In large data sets, the function head() enables you to show the first observations of a data frames. Similarly, the function tail() prints out the last observations in your data set.
head(my_dataframe)
tail(my_dataframe)
|
Patient
|
Treatment
|
Response
|
|
1
|
Psychotherapy
|
17.27828
|
|
2
|
Psychotherapy
|
28.21585
|
|
3
|
Psychotherapy
|
32.59044
|
|
4
|
Psychotherapy
|
26.51350
|
|
5
|
Psychotherapy
|
24.78736
|
|
6
|
Psychotherapy
|
23.02354
|
|
|
Patient
|
Treatment
|
Response
|
|
195
|
195
|
Medication
|
24.87308
|
|
196
|
196
|
Medication
|
33.62701
|
|
197
|
197
|
Medication
|
23.87150
|
|
198
|
198
|
Medication
|
26.41861
|
|
199
|
199
|
Medication
|
21.87547
|
|
200
|
200
|
Medication
|
14.81812
|
Similar to vectors and matrices, brackets [] are used to selects data from rows and columns in data.frames:
my_dataframe[35, 3]
## [1] 28.23622
Exercise
How can we get all columns, but only for the first 10 participants?
my_dataframe[1:10, ]
|
Patient
|
Treatment
|
Response
|
|
1
|
Psychotherapy
|
17.27828
|
|
2
|
Psychotherapy
|
28.21585
|
|
3
|
Psychotherapy
|
32.59044
|
|
4
|
Psychotherapy
|
26.51350
|
|
5
|
Psychotherapy
|
24.78736
|
|
6
|
Psychotherapy
|
23.02354
|
|
7
|
Psychotherapy
|
27.18800
|
|
8
|
Psychotherapy
|
25.35790
|
|
9
|
Psychotherapy
|
27.90346
|
|
10
|
Psychotherapy
|
16.16534
|
How to get only the Response column for all participants?
my_dataframe[ , 3]
## [1] 17.27828 28.21585 32.59044 26.51350 24.78736 23.02354 27.18800
## [8] 25.35790 27.90346 16.16534 29.02583 28.42899 26.01768 21.17097
## [15] 22.96488 32.51822 27.86215 23.76444 27.19453 31.21574 24.01656
## [22] 32.85923 30.19267 33.65342 33.16264 22.59811 35.49531 18.65914
## [29] 19.38852 40.35824 30.15550 33.36147 34.10666 34.67872 28.23622
## [36] 35.79737 37.03054 26.95119 35.00379 23.94121 30.76022 32.92460
## [43] 19.79349 28.87743 26.63434 36.87437 29.37386 32.93259 37.34053
## [50] 31.00992 33.45738 25.90962 32.16475 27.87158 33.60849 28.66175
## [57] 30.50260 28.11336 28.97617 33.47659 31.68984 31.02256 32.05707
## [64] 32.66971 28.75199 37.12150 32.53307 27.72957 30.72395 34.78095
## [71] 30.79057 26.94107 42.73528 19.17643 25.12155 32.51319 31.16573
## [78] 28.71231 36.67986 31.79341 30.08395 22.65188 24.25256 32.77380
## [85] 27.13873 22.28407 26.29213 24.81352 30.27068 36.91583 34.30821
## [92] 36.14205 25.57118 27.38895 35.85694 24.19565 28.70311 26.51289
## [99] 22.21769 29.67971 27.95608 22.68031 26.65177 28.46035 29.14500
## [106] 29.39812 29.21597 23.10617 25.02895 31.48715 19.54007 31.54108
## [113] 25.01488 22.84381 25.93818 22.49451 27.22332 27.32105 28.18441
## [120] 20.30164 18.08104 22.97129 16.59963 28.07147 29.47963 21.57659
## [127] 23.63856 32.57585 33.69816 29.42011 24.65821 23.36800 32.03514
## [134] 29.55978 23.30426 28.47392 20.83691 35.21757 19.85448 24.60950
## [141] 28.23138 30.30707 19.24222 24.86249 34.89545 17.71839 28.50023
## [148] 26.77603 26.65438 26.37316 20.28677 21.37053 23.90232 22.02737
## [155] 25.99456 25.15348 18.44601 10.20166 23.38740 22.30576 22.80130
## [162] 30.91630 26.14282 30.27141 26.23709 29.85874 22.14357 24.83422
## [169] 26.60009 27.47538 23.35859 32.08855 24.45984 28.62049 22.16793
## [176] 23.18288 18.24877 34.08925 25.57914 25.38059 25.98955 33.63981
## [183] 29.24742 20.23572 27.44383 32.20631 30.21943 20.38610 23.88014
## [190] 18.09621 23.97639 31.17328 24.65916 23.68097 24.87308 33.62701
## [197] 23.87150 26.41861 21.87547 14.81812
Another easier way for selecting particular items is using their names that is more helpful than number of the rows in large data sets:
my_dataframe[ , "Response"]
# OR:
my_dataframe$Response
Data Cleaning
Now, suppose we tested 141 students. First, let’s read and check the uncleaned data:
# read the raw data
raw_data <- read_csv(here("raw_data","raw_argumentative_exp1.csv"))
head(raw_data)
|
end_date
|
status
|
ip_address
|
progress
|
duration_in_seconds
|
subject
|
recorded_date
|
response_id
|
location_latitude
|
location_longitude
|
distribution_channel
|
user_language
|
consent_form
|
age
|
gender
|
stage1_simple
|
stage2_simple
|
stage3_simple
|
stage4_simple
|
stage5_simple
|
stage6_simple
|
stage7_simple
|
stage1_complex
|
stage2_complex
|
stage3_complex
|
stage4_complex
|
stage5_complex
|
stage6_complex
|
stage7_complex
|
thinking1
|
thinking2
|
thinking3
|
openminded1
|
openminded2
|
openminded3
|
openminded4
|
openminded5
|
openminded6
|
openminded7
|
openminded8
|
group
|
numeracy_total
|
reasoning_total
|
|
24/9/20 22:02
|
IP Address
|
202.7.193.64
|
100
|
1517
|
subj1
|
24/9/20 22:02
|
R_1f298znjmVzcOjp
|
-33.85910
|
151.2002
|
anonymous
|
EN
|
I consent
|
18
|
Female
|
|
|
|
|
|
|
|
36
|
70
|
68
|
54
|
51
|
43
|
41
|
8
|
50
|
20
|
5
|
5
|
5
|
5
|
5
|
6
|
5
|
3
|
Complex
|
9
|
9
|
|
25/9/20 3:23
|
IP Address
|
220.245.220.94
|
100
|
1131
|
subj2
|
25/9/20 3:23
|
R_tL0A9P33Gi18I0N
|
-34.03680
|
150.6672
|
anonymous
|
EN
|
I consent
|
18
|
Male
|
50
|
55
|
55
|
90
|
75
|
50
|
35
|
|
|
|
|
|
|
|
8
|
10
|
39
|
6
|
6
|
6
|
5
|
6
|
6
|
5
|
6
|
Simple
|
9
|
10
|
|
27/9/20 22:59
|
IP Address
|
121.210.0.211
|
100
|
709
|
subj3
|
27/9/20 22:59
|
R_1LNyJhCKxTAAMOW
|
-33.85910
|
151.2002
|
anonymous
|
EN
|
I consent
|
19
|
Female
|
50
|
50
|
77
|
60
|
25
|
20
|
13
|
|
|
|
|
|
|
|
8
|
50
|
20
|
6
|
5
|
4
|
5
|
5
|
6
|
5
|
6
|
Simple
|
10
|
8
|
|
27/9/20 23:18
|
IP Address
|
58.179.100.109
|
100
|
949
|
subj4
|
27/9/20 23:18
|
R_3enxzUsEYgs5r1a
|
-12.63921
|
141.8741
|
anonymous
|
EN
|
I consent
|
27
|
Female
|
|
|
|
|
|
|
|
70
|
80
|
90
|
95
|
70
|
80
|
90
|
8
|
50
|
20
|
6
|
6
|
6
|
1
|
6
|
6
|
6
|
1
|
Complex
|
8
|
7
|
|
28/9/20 0:45
|
IP Address
|
120.154.53.68
|
100
|
1097
|
subj5
|
28/9/20 0:45
|
R_2Qzl2096a4KNE29
|
-33.85910
|
151.2002
|
anonymous
|
EN
|
I consent
|
19
|
Male
|
71
|
73
|
85
|
95
|
95
|
32
|
32
|
|
|
|
|
|
|
|
4
|
10
|
39
|
6
|
6
|
5
|
5
|
6
|
6
|
6
|
6
|
Simple
|
11
|
11
|
|
28/9/20 2:20
|
IP Address
|
1.129.107.6
|
100
|
880
|
subj6
|
28/9/20 2:20
|
R_esb71WOTQySjusF
|
-33.85910
|
151.2002
|
anonymous
|
EN
|
I consent
|
20
|
Female
|
|
|
|
|
|
|
|
89
|
100
|
44
|
55
|
100
|
50
|
55
|
8
|
50
|
20
|
6
|
6
|
6
|
5
|
6
|
6
|
6
|
6
|
Complex
|
10
|
10
|
Now, let’s do some cleanining using dplyr, tidyr and other tidyverse libraries. Finally, we will check the data:
cleaned_data <- raw_data %>%
filter(progress == 100) %>% # filter out unfinished participants
select(-end_date, -status,-ip_address, -duration_in_seconds, -recorded_date:-user_language) %>% #remove some useless columns
mutate(openminded_total= openminded1+openminded2+openminded3+openminded4+openminded5+openminded6+openminded7+openminded8) %>%# create a total score for our questionnaire
mutate(thinking1= case_when(thinking1=='4'~ 1,T~0),
thinking2= case_when(thinking2=='10'~ 1,T~0),
thinking3= case_when(thinking3=='39'~ 1,T~0),
thinking_total= thinking1 + thinking2 + thinking3) %>%
select(-thinking1:-openminded8) %>%
pivot_longer(cols = c(stage1_simple:stage7_simple,stage1_complex:stage7_complex),names_to = 'stage',values_to = 'truth_estimate') %>% # make our dataframe long
#pivot_wider(names_from = stage, values_from= truth_estimate) # this code change our dataframe back to wide
filter(!is.na(truth_estimate)) %>% #remove rows with truth_estimate == NA
mutate(stage= gsub("_.*", "", stage)) %>%
rename(consent= consent_form) %>% # rename a column
#mutate_if(is.character, factor) %>%
mutate(subject= factor(subject), # convert all characters to factor
group = factor(group),
stage = factor(stage))
|
progress
|
subject
|
consent
|
age
|
gender
|
group
|
numeracy_total
|
reasoning_total
|
openminded_total
|
thinking_total
|
stage
|
truth_estimate
|
|
100
|
subj1
|
I consent
|
18
|
Female
|
Complex
|
9
|
9
|
39
|
0
|
stage1
|
36
|
|
100
|
subj1
|
I consent
|
18
|
Female
|
Complex
|
9
|
9
|
39
|
0
|
stage2
|
70
|
|
100
|
subj1
|
I consent
|
18
|
Female
|
Complex
|
9
|
9
|
39
|
0
|
stage3
|
68
|
|
100
|
subj1
|
I consent
|
18
|
Female
|
Complex
|
9
|
9
|
39
|
0
|
stage4
|
54
|
|
100
|
subj1
|
I consent
|
18
|
Female
|
Complex
|
9
|
9
|
39
|
0
|
stage5
|
51
|
|
100
|
subj1
|
I consent
|
18
|
Female
|
Complex
|
9
|
9
|
39
|
0
|
stage6
|
43
|
Ok, now the data is clean and tidy which means:
- Each variable forms a column.
- Each observation forms a row.
- Each type of observational unit forms a table (Wickham, 2014).
Check the dataframe and all the data types:
str(cleaned_data)
## tibble [917 × 12] (S3: tbl_df/tbl/data.frame)
## $ progress : num [1:917] 100 100 100 100 100 100 100 100 100 100 ...
## $ subject : Factor w/ 131 levels "subj1","subj10",..: 1 1 1 1 1 1 1 45 45 45 ...
## $ consent : chr [1:917] "I consent" "I consent" "I consent" "I consent" ...
## $ age : num [1:917] 18 18 18 18 18 18 18 18 18 18 ...
## $ gender : chr [1:917] "Female" "Female" "Female" "Female" ...
## $ group : Factor w/ 2 levels "Complex","Simple": 1 1 1 1 1 1 1 2 2 2 ...
## $ numeracy_total : num [1:917] 9 9 9 9 9 9 9 9 9 9 ...
## $ reasoning_total : num [1:917] 9 9 9 9 9 9 9 10 10 10 ...
## $ openminded_total: num [1:917] 39 39 39 39 39 39 39 46 46 46 ...
## $ thinking_total : num [1:917] 0 0 0 0 0 0 0 2 2 2 ...
## $ stage : Factor w/ 7 levels "stage1","stage2",..: 1 2 3 4 5 6 7 1 2 3 ...
## $ truth_estimate : num [1:917] 36 70 68 54 51 43 41 50 55 55 ...
Finally, we save our data to the cleaned_data folder.
write_csv(cleaned_data, here("cleaned_data","argumentative_exp1.csv"))
Descriptive Statistics
Note: All the data that we use here is manipulated (fabricated) for teaching purpuses. In our study, we failed to find such beautiful and interesting results.
Now, let’s do some descriptive statistics. First, we can open a new script called analysis_exp1.r and read the cleaned data again.
data_exp1 <- read_csv(here("cleaned_data","argumentative_exp1.csv"))
How many participants in total?
data_exp1 %>% summarise(n= n_distinct(subject))
how many participants in each group?
data_exp1 %>%
group_by(subject) %>%
filter(row_number()==1) %>%
ungroup () %>%
group_by(group) %>%
count()
|
group
|
n
|
|
Complex
|
65
|
|
Simple
|
66
|
Find the mean and sd for numeric variables using base R summary function:
data_exp1 %>%
group_by(subject) %>%
filter(row_number()==1) %>%
ungroup () %>%
summary()
## progress subject consent age
## Min. :100 Length:131 Length:131 Min. :16.00
## 1st Qu.:100 Class :character Class :character 1st Qu.:18.00
## Median :100 Mode :character Mode :character Median :19.00
## Mean :100 Mean :21.15
## 3rd Qu.:100 3rd Qu.:20.00
## Max. :100 Max. :63.00
## gender group numeracy_total reasoning_total
## Length:131 Length:131 Min. : 0.000 Min. : 4.00
## Class :character Class :character 1st Qu.: 8.000 1st Qu.: 8.00
## Mode :character Mode :character Median :10.000 Median : 8.00
## Mean : 8.779 Mean : 8.45
## 3rd Qu.:10.000 3rd Qu.: 9.00
## Max. :11.000 Max. :12.00
## openminded_total thinking_total stage truth_estimate
## Min. :19.0 Min. :0.0000 Length:131 Min. : 0.00
## 1st Qu.:33.5 1st Qu.:0.0000 Class :character 1st Qu.: 43.50
## Median :39.0 Median :0.0000 Mode :character Median : 61.00
## Mean :38.2 Mean :0.8092 Mean : 57.09
## 3rd Qu.:43.0 3rd Qu.:1.0000 3rd Qu.: 75.50
## Max. :48.0 Max. :3.0000 Max. :100.00
Alternatively, we can use base Rsummaryfunctionskimr` library:
data_exp1 %>%
group_by(subject) %>%
filter(row_number()==1) %>%
ungroup () %>%
dplyr::select (age, numeracy_total, reasoning_total, openminded_total, thinking_total) %>%
skimr::skim()
|
skim_type
|
skim_variable
|
n_missing
|
complete_rate
|
numeric.mean
|
numeric.sd
|
numeric.p0
|
numeric.p25
|
numeric.p50
|
numeric.p75
|
numeric.p100
|
numeric.hist
|
|
numeric
|
age
|
0
|
1
|
21.1526718
|
6.515630
|
16
|
18.0
|
19
|
20
|
63
|
▇▁▁▁▁
|
|
numeric
|
numeracy_total
|
0
|
1
|
8.7786260
|
2.274576
|
0
|
8.0
|
10
|
10
|
11
|
▁▁▁▂▇
|
|
numeric
|
reasoning_total
|
0
|
1
|
8.4503817
|
1.683466
|
4
|
8.0
|
8
|
9
|
12
|
▁▅▇▆▃
|
|
numeric
|
openminded_total
|
0
|
1
|
38.1984733
|
6.153698
|
19
|
33.5
|
39
|
43
|
48
|
▁▂▇▇▆
|
|
numeric
|
thinking_total
|
0
|
1
|
0.8091603
|
1.038598
|
0
|
0.0
|
0
|
1
|
3
|
▇▃▁▂▂
|
Exercise
For this exercise, we use a dataset of one of my own studies. In this study, we asked participants to guess the physical brightness of reasoning arguments and then we gave a cognitive ability test. (See the original study here). Open ghasemi_brightness_exp4.csv file and answer to the following questions:
- How many participants did we test in total?
- Find out how many male and female we tested.
- Calculate mean and sd for age and cognitive ability (
cog_ability).
ghasemi_data <- read_csv(here("cleaned_data","ghasemi_brightness_exp4.csv"))
ghasemi_data %>% summarise(n = n_distinct(participant)) # number of participants:200
ghasemi_data %>% group_by (participant) %>% filter (row_number()==1) %>% group_by (gender) %>% summarise(n= n()) %>% ungroup() # 183 female, 17 male
|
gender
|
n
|
|
Female
|
183
|
|
Male
|
17
|
ghasemi_data %>% dplyr::select (age, cog_ability) %>% skimr::skim() # mean and sd for age and cognitive ability
Data summary
|
|
|
|
Name
|
Piped data
|
|
Number of rows
|
38400
|
|
Number of columns
|
2
|
|
_______________________
|
|
|
Column type frequency:
|
|
|
numeric
|
2
|
|
________________________
|
|
|
Group variables
|
|
Variable type: numeric
|
skim_variable
|
n_missing
|
complete_rate
|
mean
|
sd
|
p0
|
p25
|
p50
|
p75
|
p100
|
hist
|
|
age
|
0
|
1
|
22.20
|
6.78
|
17
|
19
|
20
|
22
|
52
|
▇▁▁▁▁
|
|
cog_ability
|
0
|
1
|
39.55
|
9.46
|
11
|
34
|
40
|
46
|
61
|
▁▃▇▆▂
|
Data Visualization
First, we need to create a dataset with aggregated truth estimate scores over group and stage. We will use this dataset for line and bar graphs.
aggregated_data_exp1 <- data_exp1 %>%
group_by(stage, group) %>%
mutate(truth_estimate = mean(truth_estimate)) %>%
ungroup()
barplot_exp1 <- aggregated_data_exp1 %>%
ggplot(aes(x=stage, y= truth_estimate, fill=group)) +
geom_bar(stat = "identity", position= "dodge")+
# stat_summary(fun= mean, geom = "bar", position = "dodge")+ # can be used instead of geom_bar() for long dataframes
labs (x= '', y= "Truth Likelihhod Estimate") +
theme_bw() +
scale_fill_jama()
barplot_exp1

barplot_facet_exp1 <- aggregated_data_exp1 %>%
ggplot(aes(x=group, y= truth_estimate, fill=stage)) +
geom_bar(stat = "identity", position= "dodge")+
labs (x= '', y= "Truth Likelihhod Estimate") +
theme_bw() +
theme(legend.position = "none",
axis.text=element_text(size=11),
axis.title = element_text(size = 12)) +
facet_wrap(~stage)+
scale_fill_jco()
barplot_facet_exp1

lineplot_exp1 <- aggregated_data_exp1 %>%
ggplot(aes(x=factor(stage), y= truth_estimate, group= group, color= group)) +
geom_line(aes(linetype= group)) +
geom_point(size= 5)+
labs (x= '', y= "Truth Likelihhod Estimate") +
theme_classic() +
theme(legend.position = "bottom",
axis.text=element_text(size=11),
axis.title = element_text(size = 12)) +
scale_color_nejm()
lineplot_exp1

violinplot_exp1 <- data_exp1 %>%
ggplot(aes(x=factor(stage), y= truth_estimate, fill= group)) +
geom_violin()+
labs (x= '', y= "Truth Likelihhod Estimate") +
theme_bw() +
theme(legend.position = "bottom",
axis.text=element_text(size=11),
axis.title = element_text(size = 12)) +
scale_fill_d3()
violinplot_exp1

boxplot_exp1 <- data_exp1 %>%
ggplot(aes(x=factor(stage), y= truth_estimate, fill= group)) +
geom_boxplot()+
#geom_point(position = position_dodge(width=0.75), alpha= .5)+
labs (x= '', y= "Truth Likelihhod Estimate") +
theme_bw() +
theme(legend.position = "bottom",
axis.text=element_text(size=11),
axis.title = element_text(size = 12)) +
scale_fill_simpsons()
boxplot_exp1

boxplot_facet_exp1 <- data_exp1 %>%
ggplot(aes(x=factor(stage), y= truth_estimate, fill= group)) +
geom_boxplot()+
labs (x= '', y= "Truth Likelihhod Estimate") +
theme_bw() +
theme(legend.position = "bottom",
axis.text=element_text(size=11),
axis.title = element_text(size = 12),
axis.text.x = element_text(angle = 90, vjust = 0.5, hjust=1)) +
facet_wrap(~group)+
scale_color_simpsons()
boxplot_facet_exp1

How to combine multiple plots? We can use the patchwork package. A nice tutorial on using this package can be found here
combined_plot_exp1 <- (barplot_facet_exp1+lineplot_exp1) / (violinplot_exp1+boxplot_exp1)
combined_plot_exp1

How to save a plot?
ggsave(combined_plot_exp1, filename = here("outputs","combined_plot_exp1.png"), dpi=300)
Data Analysis
t-test
Is there a difference between groups at the first stage? Ideally, we want participants’ ratings at the first stage be similar for both groups because we have not done any manipulations. Previous graphs showed us that ratings of simple and complex group at this stage are pretty close. Let’s test that using an independent t-test (because we have 2 independent groups):
# Is there a difference between groups at the first stage?
data_exp1 %>%
group_by(group) %>%
filter(stage=='stage1') %>%
ungroup () %>%
t.test(truth_estimate~group, data = ., paired=FALSE)
##
## Welch Two Sample t-test
##
## data: truth_estimate by group
## t = -0.75145, df = 104.95, p-value = 0.4541
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -11.883716 5.351781
## sample estimates:
## mean in group Complex mean in group Simple
## 55.44615 58.71212
Now, we wonder if opposing arguments were effective at all, regardless of participants’ group. So, we would like to test if ratings at the final stage are lower than ratings at the stage 4? Since a pair of score at stage 4 and stage 7 is coming from a same person, we use paired t-test.
# Is there a difference between ratings of stage4 and stage7?
data_exp1 %>%
filter(stage=='stage4' | stage=='stage7') %>%
ungroup () %>%
t.test(truth_estimate~stage, data = ., paired=TRUE)
##
## Paired t-test
##
## data: truth_estimate by stage
## t = 12.788, df = 130, p-value < 0.00000000000000022
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## 32.64368 44.59296
## sample estimates:
## mean of the differences
## 38.61832
Exercise
John et al. (2019) investigated the consequences of backing down (changing one’s mind in lights of evidence)and how other people view someone who change their mind. In their second experiments, they presented participants either with a person who changes their mind or a person who refuses to back down. Then, they asked participants to rate how intelligent and confident the person is (See the original study here). They reported that:
“Relative to the entrepreneur who did not back down, participants judged the entrepreneur who backed down as more intelligent (M_backed_down=5.13 out of 7, SD=1.09; M_did_not_back_down=3.97, SD=1.54; t(271.12)=−7.59, p < .001) but less confident (M_backed_down=4.50 out of 7, SD=1.36; M_did_not_back_down=5.65, SD=1.10; t(291.01)=8.08, p < .001).”.
Open the john_backdown_exp2.csv file and try to reproduce their results. Run two separate independent t-test, one with intelligent as the dependent variable and one with confident as the dependent variable. For both t-test, use back_down as the between-subject independent variable.
john_data <- read_csv(here("cleaned_data","john_backdown_exp2.csv"))
t.test(intelligent~back_down, data = john_data, paired=FALSE)
##
## Welch Two Sample t-test
##
## data: intelligent by back_down
## t = 7.5853, df = 271.12, p-value = 0.0000000000005319
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## 0.8577107 1.4590076
## sample estimates:
## mean in group backed_down mean in group did_not_back_down
## 5.129412 3.971053
t.test(confident~back_down, data = john_data, paired=FALSE)
##
## Welch Two Sample t-test
##
## data: confident by back_down
## t = -8.0763, df = 291.01, p-value = 0.00000000000001787
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -1.4257768 -0.8670294
## sample estimates:
## mean in group backed_down mean in group did_not_back_down
## 4.503268 5.649671
Analysis of Variance (ANOVA)
Now, let’s answer our main question: Do participants in the simple group show higher ratings for supportive arguments (stage 2 to 4) and lower ratings for opposing arguments (stage 5 to 7), compared to participants in the complex group? If this is the case. we expect an interaction in the traditional Analysis of Variance (AONVA) test.
aov_m1 <- aov_car (truth_estimate ~ group*stage +
Error(subject/stage), data = data_exp1)
|
Effect
|
df
|
MSE
|
F
|
ges
|
p.value
|
|
group
|
1, 129
|
949.04
|
0.01
|
<.0001
|
.94
|
|
stage
|
4.45, 574.05
|
515.69
|
59.48 ***
|
.25
|
<.0001
|
|
group:stage
|
4.45, 574.05
|
515.69
|
13.34 ***
|
.07
|
<.0001
|
As you can see, we found a significant main effect of stage and a significant group by stage interaction. We can use the emmeans package to do post-hoc tests.
# main effect of stage
emmeans(aov_m1, 'stage')
## stage emmean SE df lower.CL upper.CL
## stage1 57.1 1.88 763 53.4 60.8
## stage2 66.8 1.88 763 63.1 70.5
## stage3 74.6 1.88 763 70.9 78.3
## stage4 79.6 1.88 763 75.9 83.3
## stage5 62.4 1.88 763 58.7 66.1
## stage6 52.5 1.88 763 48.9 56.2
## stage7 41.1 1.88 763 37.4 44.8
##
## Results are averaged over the levels of: group
## Warning: EMMs are biased unless design is perfectly balanced
## Confidence level used: 0.95
pairs(emmeans(aov_m1, 'stage'), adjust= 'holm')
## contrast estimate SE df t.ratio p.value
## stage1 - stage2 -9.74 2.42 774 -4.031 0.0004
## stage1 - stage3 -17.53 2.42 774 -7.256 <.0001
## stage1 - stage4 -22.50 2.42 774 -9.311 <.0001
## stage1 - stage5 -5.29 2.42 774 -2.187 0.1160
## stage1 - stage6 4.53 2.42 774 1.876 0.1220
## stage1 - stage7 15.98 2.42 774 6.613 <.0001
## stage2 - stage3 -7.79 2.42 774 -3.225 0.0066
## stage2 - stage4 -12.76 2.42 774 -5.280 <.0001
## stage2 - stage5 4.46 2.42 774 1.844 0.1220
## stage2 - stage6 14.28 2.42 774 5.908 <.0001
## stage2 - stage7 25.72 2.42 774 10.644 <.0001
## stage3 - stage4 -4.97 2.42 774 -2.055 0.1206
## stage3 - stage5 12.25 2.42 774 5.069 <.0001
## stage3 - stage6 22.07 2.42 774 9.132 <.0001
## stage3 - stage7 33.51 2.42 774 13.869 <.0001
## stage4 - stage5 17.22 2.42 774 7.124 <.0001
## stage4 - stage6 27.04 2.42 774 11.188 <.0001
## stage4 - stage7 38.48 2.42 774 15.924 <.0001
## stage5 - stage6 9.82 2.42 774 4.064 0.0004
## stage5 - stage7 21.27 2.42 774 8.800 <.0001
## stage6 - stage7 11.45 2.42 774 4.736 <.0001
##
## Results are averaged over the levels of: group
## P value adjustment: holm method for 21 tests
# group by stage interaction
emmeans(aov_m1, "group", by= "stage")
## stage = stage1:
## group emmean SE df lower.CL upper.CL
## Complex 55.4 2.67 766 50.2 60.7
## Simple 58.7 2.65 761 53.5 63.9
##
## stage = stage2:
## group emmean SE df lower.CL upper.CL
## Complex 63.3 2.67 766 58.1 68.6
## Simple 70.3 2.65 761 65.1 75.5
##
## stage = stage3:
## group emmean SE df lower.CL upper.CL
## Complex 70.0 2.67 766 64.7 75.2
## Simple 79.3 2.65 761 74.1 84.5
##
## stage = stage4:
## group emmean SE df lower.CL upper.CL
## Complex 71.6 2.67 766 66.3 76.8
## Simple 87.6 2.65 761 82.4 92.8
##
## stage = stage5:
## group emmean SE df lower.CL upper.CL
## Complex 64.2 2.67 766 58.9 69.4
## Simple 60.5 2.65 761 55.3 65.8
##
## stage = stage6:
## group emmean SE df lower.CL upper.CL
## Complex 57.9 2.67 766 52.7 63.2
## Simple 47.2 2.65 761 41.9 52.4
##
## stage = stage7:
## group emmean SE df lower.CL upper.CL
## Complex 51.1 2.67 766 45.9 56.4
## Simple 31.1 2.65 761 25.9 36.3
##
## Warning: EMMs are biased unless design is perfectly balanced
## Confidence level used: 0.95
update(pairs(emmeans(aov_m1, "group", by= "stage")), by = NULL, adjust = "holm")
## contrast stage estimate SE df t.ratio p.value
## Complex - Simple stage1 -3.27 3.76 763 -0.868 0.6673
## Complex - Simple stage2 -6.96 3.76 763 -1.851 0.1935
## Complex - Simple stage3 -9.29 3.76 763 -2.469 0.0550
## Complex - Simple stage4 -16.02 3.76 763 -4.259 0.0001
## Complex - Simple stage5 3.64 3.76 763 0.967 0.6673
## Complex - Simple stage6 10.79 3.76 763 2.868 0.0213
## Complex - Simple stage7 20.08 3.76 763 5.337 <.0001
##
## P value adjustment: holm method for 7 tests
You can use the afex_plot function from afex to create beautiful plots. Those plots interacts nicely with ggplot:
afex_plot(aov_m1, x = "stage", trace = "group", error='between',
line_arg = list(size=1),
point_arg = list(size=3.5),
data_arg = list(size= 1, color= 'grey', width=.4),
data_geom = geom_boxplot,
mapping = c("linetype", "shape", "fill"),
legend_title = "Group") +
labs(y = "Truth Likelihhod Estimate", x = "") +
theme_bw()+ # remove the grey background and grid
theme(axis.text=element_text(size=13),
axis.title = element_text(size = 13),
legend.text=element_text(size=13),
legend.title=element_text(size=13),
legend.position='bottom',
legend.key.size = unit(1, "cm"),
legend.background = element_rect(colour = 'black', fill = 'white', linetype='solid'))+
scale_color_simpsons() +
scale_fill_simpsons()

If you are interested in this topic, check out this nice tutorial about using afex to run ANOVA, and also this interesting tutorial on the emmeans package.
Exercise
Rotello et al. (2018) investigated the association between the race (White vs. Black faces) and the gun-tool judgments. In their first experiments, they presented participants with 16 White male faces and 16 Black male faces, and following that 8 images of guns and 8 images of tools. They asked participants to judge if the object is a tool or a gun by pressing keyboard buttons. Then, they ran an ANOVA to see if participants’ gun responses are higher for any of the races. So, they included prime race (Black, White) and target identity (gun, tool) as independent variables and participants’ gun responses as dependent variable into their linear model (See the original study here). They found that:
“Participants made more gun responses to guns than to tools, F(1,45) = 53243, p < 0.0001, η2g = 0.998. However, the race of the prime face did not matter, F(1,45) = 0.287, p > 0.59, η2g = 0.001, nor was there an interaction of prime race with target object, F(1,45) = 0.022, p > 0.88, η2g = 0.000)”.
Open the rotello_shooter_exp1.csv file and try to reproduce their results. Run an ANOVA (type III) with resp as the dependent variable and target, prime, and their interaction as independent variables.
# load the general data file
rotello_data <- read_csv(here("cleaned_data","rotello_shooter_exp1.csv"))
# ANOVA
rotello_aov <- aov_car (resp ~ target*prime +
Error(subject/target*prime), data = rotello_data)
|
Effect
|
df
|
MSE
|
F
|
ges
|
p.value
|
|
target
|
1, 45
|
0.00
|
53242.99 ***
|
>.99
|
<.0001
|
|
prime
|
1, 45
|
0.00
|
0.29
|
.001
|
.59
|
|
target:prime
|
1, 45
|
0.00
|
0.02
|
<.0001
|
.88
|
Correlation
Now, let’s answer to another question of this study: does persuasion and dissuasion is related to open-mindedness, cognitive ability, reasoning abilities, and cognitive style? To answer this question, we need to create two indexes (scores) one for persuasion and one for dissuasion. Then we can do a correlation test:
cor_data_exp1 <- data_exp1 %>%
pivot_wider(names_from = stage, values_from = truth_estimate) %>%
group_by(subject) %>%
mutate(persuasion_index= stage2+ stage3+ stage4 - stage1,
dissuasion_index= (101-stage5) + (101-stage6) + (101-stage7) - (101-stage4)) %>%
ungroup()%>%
dplyr::select(persuasion_index,dissuasion_index,openminded_total,numeracy_total,thinking_total,reasoning_total)
#---------- Base R:
cor(cor_data_exp1, method = "pearson", use = "complete.obs")
#---------- Psych library:
cor_data_exp1 %>%
psych::pairs.panels(method = "pearson", hist.col = "#00AFBB", density = T, ellipses = F, stars = T)
#---------- Correlation library:
correlation::correlation(cor_data_exp1) %>% summary()
#---------- apaTables library:
cor_data_exp1 %>%
apaTables::apa.cor.table(filename="./outputs/CorMatrix.doc", show.conf.interval=T)
|
|
persuasion_index
|
dissuasion_index
|
openminded_total
|
numeracy_total
|
thinking_total
|
reasoning_total
|
|
persuasion_index
|
1.00
|
0.26
|
0.25
|
0.16
|
0.16
|
0.11
|
|
dissuasion_index
|
0.26
|
1.00
|
-0.03
|
-0.03
|
-0.09
|
0.15
|
|
openminded_total
|
0.25
|
-0.03
|
1.00
|
0.40
|
0.26
|
0.11
|
|
numeracy_total
|
0.16
|
-0.03
|
0.40
|
1.00
|
0.44
|
0.15
|
|
thinking_total
|
0.16
|
-0.09
|
0.26
|
0.44
|
1.00
|
0.29
|
|
reasoning_total
|
0.11
|
0.15
|
0.11
|
0.15
|
0.29
|
1.00
|
|
Parameter
|
reasoning_total
|
thinking_total
|
numeracy_total
|
openminded_total
|
dissuasion_index
|
|
persuasion_index
|
0.11
|
0.16
|
0.16
|
0.25
|
0.26
|
|
dissuasion_index
|
0.15
|
-0.09
|
-0.03
|
-0.03
|
|
|
openminded_total
|
0.11
|
0.26
|
0.40
|
|
|
|
numeracy_total
|
0.15
|
0.44
|
|
|
|
|
thinking_total
|
0.29
|
|
|
|
|
Exercise
Pennycook et al. (2020) investigated the relationship between actively open-minded thinking style about evidence (AOT-E) and different political, scientific, and religious beliefs (see the original paper here). In their first experiment, they calculated the correlation of AOTE and scientific beliefs items (global warming, evolution, etc.) and they found the following results:
Open the pennycook_aote_exp1.csv file and try to reproduce their results by creating the same correlation matrix.
pennycook_data <- read_csv(here("cleaned_data","pennycook_aote_exp1.csv"))
#---------- Base R:
cor(pennycook_data, method = "pearson", use = "complete.obs")
#---------- Psych library:
pennycook_data %>%
psych::pairs.panels(method = "pearson", hist.col = "#00AFBB", density = T, ellipses = F, stars = T)
#---------- Correlation library:
correlation::correlation(pennycook_data) %>% summary()
#---------- apaTables library:
pennycook_data %>%
apaTables::apa.cor.table(filename="./outputs/CorMatrix.doc", show.conf.interval=T)
|
Parameter
|
trust_scien
|
gm_health
|
tech_problems
|
modern_medicine
|
old_earth
|
vaccines
|
stem_cell
|
big_bang
|
evolution
|
global_warming
|
|
aote
|
0.35
|
0.36
|
0.44
|
0.33
|
0.40
|
0.47
|
0.45
|
0.51
|
0.51
|
0.37
|
|
global_warming
|
0.42
|
0.06
|
0.14
|
0.18
|
0.33
|
0.26
|
0.31
|
0.33
|
0.38
|
|
|
evolution
|
0.48
|
0.33
|
0.28
|
0.36
|
0.47
|
0.39
|
0.54
|
0.78
|
|
|
|
big_bang
|
0.49
|
0.37
|
0.28
|
0.36
|
0.45
|
0.37
|
0.54
|
|
|
|
|
stem_cell
|
0.47
|
0.34
|
0.36
|
0.47
|
0.40
|
0.40
|
|
|
|
|
|
vaccines
|
0.43
|
0.52
|
0.49
|
0.53
|
0.38
|
|
|
|
|
|
|
old_earth
|
0.29
|
0.24
|
0.21
|
0.33
|
|
|
|
|
|
|
|
modern_medicine
|
0.43
|
0.42
|
0.47
|
|
|
|
|
|
|
|
|
tech_problems
|
0.33
|
0.39
|
|
|
|
|
|
|
|
|
|
gm_health
|
0.31
|
|
|
|
|
|
|
|
|
|
Linear Regression
In the previous section, we found that open-mindedness (AOT-E) is correlated with persuasion. Now, one may ask if open-mindedness can predict persuasion after controlling for reasoning and controlling abilities? To answer that, we can run a multiple regression analysis:
exp1_reg=lm(persuasion_index ~ openminded_total+ numeracy_total+ thinking_total+ reasoning_total,
data=cor_data_exp1)
|
term
|
estimate
|
std.error
|
statistic
|
p.value
|
|
(Intercept)
|
78.57
|
33.08
|
2.38
|
0.02
|
|
openminded_total
|
1.62
|
0.72
|
2.23
|
0.03
|
|
numeracy_total
|
0.72
|
2.11
|
0.34
|
0.73
|
|
thinking_total
|
3.09
|
4.51
|
0.68
|
0.49
|
|
reasoning_total
|
1.77
|
2.52
|
0.70
|
0.48
|
Exercise
Trémolière and Djeriouat (2020) examined the role of cognitive reflection and belief in science in climate change skepticism. In their first study, they revealed that cognitive reflection and belief in science negetively predicted climate change skepticism even after controlling for demographic and cognitive ability variables (see the original paper here).
Open the tremoliere_data_exp1.csv file and try to reproduce their results by running a multiple linear regression. Enter age, gender, education, belief in science, literacy, numeracy (Numtotal), and cognitive reflection as predictors and enter climate change skepticism (climato) as the outcome variable.
Tremoliere_data <- read_csv(here("cleaned_data","tremoliere_data_exp1.csv"))
Tremoliere_reg=lm(Climato ~ Age+ Gender+ Education+ BeliefInSciencetotal+ Literacy+ Numtotal+ CognitiveReflection,
data=Tremoliere_data)
|
term
|
estimate
|
std.error
|
statistic
|
p.value
|
|
(Intercept)
|
57.57
|
5.19
|
11.09
|
0.00
|
|
Age
|
0.01
|
0.05
|
0.24
|
0.81
|
|
Gender
|
-5.68
|
1.34
|
-4.23
|
0.00
|
|
Education
|
0.54
|
0.38
|
1.43
|
0.15
|
|
BeliefInSciencetotal
|
-0.20
|
0.06
|
-3.62
|
0.00
|
|
Literacy
|
-0.49
|
0.51
|
-0.96
|
0.34
|
|
Numtotal
|
-1.52
|
0.83
|
-1.82
|
0.07
|
|
CognitiveReflection
|
-18.58
|
4.26
|
-4.37
|
0.00
|
|
r.squared
|
adj.r.squared
|
sigma
|
statistic
|
p.value
|
df
|
logLik
|
AIC
|
BIC
|
deviance
|
df.residual
|
nobs
|
|
0.19
|
0.17
|
12.65
|
11.91
|
0
|
7
|
-1467.77
|
2953.54
|
2988.81
|
58235.89
|
364
|
372
|
Rmarkdown
To be completed…
References
Ghasemi, O., Handley, S., & Howarth, S. (2020). The Bright Homunculus in our Head: Individual Differences in Intuitive Sensitivity to Logical Validity.
John, L. K., Jeong, M., Gino, F., & Huang, L. (2019). The self-presentational consequences of upholding one’s stance in spite of the evidence. Organizational Behavior and Human Decision Processes, 154, 1-14.
Pennycook, G., Cheyne, J. A., Koehler, D. J., & Fugelsang, J. A. (2020). On the belief that beliefs should change according to evidence: Implications for conspiratorial, moral, paranormal, political, religious, and science beliefs. Judgment and Decision Making, 15(4), 476.
Rotello, C. M., Kelly, L. J., Heit, E., Vazire, S., & Vul, E. (2018). The Shape of ROC Curves in Shooter Tasks: Implications for Best Practices in Analysis. Collabra: Psychology, 4(1).
Trémolière, B., & Djeriouat, H. (2020). Don’t you see that its cold! Exploring the roles of cognitive reflection, climate science literacy, illusion of knowledge, and political orientation in climate change skepticism.
Wickham, H. (2014). Tidy data. Journal of Statistical Software, 59(10), 1-23.
LS0tCnRpdGxlOiAiSW50cm9kdWN0aW9uIHRvIFIiCmF1dGhvcjoKICAtIG5hbWU6ICJPbWlkIEdoYXNlbWkiCiAgICBhZmZpbGlhdGlvbjogTWFjcXVhcmllIFVuaXZlcnNpdHkKICAgIGVtYWlsOiBvbWlkcmV6YS5naGFzZW1pQGhkci5tcS5lZHUuYXUKICAtIG5hbWU6ICJNYWhkaSBNYXppZGkiCiAgICBhZmZpbGlhdGlvbjogVW5pdmVyc2l0eSBvZiBXZXN0ZXJuIEF1c3RyYWxpYQogICAgZW1haWw6IG1haGRpLm1hemlkaXNoYXJhZmFiYWRpQHJlc2VhcmNoLnV3YS5lZHUuYXUKZGF0ZTogImByIGZvcm1hdChTeXMudGltZSgpLCAnJWQgJUIsICVZJylgIgpvdXRwdXQ6IAogIGh0bWxfZG9jdW1lbnQ6CiAgICBrZWVwX21kOiB5ZXMKICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQogICAgdGhlbWU6IGNlcnVsZWFuCiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCiAgICAjY29kZV9mb2xkaW5nOiBoaWRlCiAgICB0b2M6IHRydWUKICAgIHRvY19mbG9hdDogdHJ1ZQogICAgZGZfcHJpbnQ6ICJrYWJsZSIKLS0tCgpUaGlzIGRvY3VtZW50IGlzIHRoZSBzdW1tYXJ5IG9mIHRoZSAqKkludHJvZHVjdGlvbiB0byBSKiogd29ya3Nob3AuIAoKQWxsIGNvcnJlc3BvbmRlbmNlIHJlbGF0ZWQgdG8gdGhpcyBkb2N1bWVudCBzaG91bGQgYmUgYWRkcmVzc2VkIHRvOiAKCjxjZW50ZXI+Ck9taWQgR2hhc2VtaSAoTWFjcXVhcmllIFVuaXZlcnNpdHksIFN5ZG5leSwgTlNXLCAyMTA5LCBBVVNUUkFMSUEpIAoKRW1haWw6IG9taWRyZXphLmdoYXNlbWlAaGRyLm1xLmVkdS5hdSAKPC9jZW50ZXI+CgoKCjxzdHlsZT4KCmJvZHl7IC8qIE5vcm1hbCAgKi8KICAgICAgZm9udC1zaXplOiAxOHB4OwogICAgICB0ZXh0LWFsaWduOiBqdXN0aWZ5OwogICAgICBsaW5lLWhlaWdodDogMS42OwogICAgICBmb250LWZhbWlseTogIlRpbWVzIE5ldyBSb21hbiIsIFRpbWVzLCBzZXJpZjsKfQpjb2RlLnJ7IC8qIENvZGUgYmxvY2sgKi8KICAgIGZvbnQtc2l6ZTogMTRweDsKfQpwcmUgeyAvKiBDb2RlIGJsb2NrIC0gZGV0ZXJtaW5lcyBjb2RlIHNwYWNpbmcgYmV0d2VlbiBsaW5lcyAqLwogICAgZm9udC1zaXplOiAxMnB4Owp9Cgo8L3N0eWxlPgoKCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQprbml0cjo6b3B0c19jaHVuayRzZXQoZWNobyA9IFRSVUUpCmtuaXRyOjpvcHRzX2NodW5rJHNldChmaWcuYWxpZ249ImNlbnRlciIpCmBgYAoKCgpgYGB7ciBsaWJyYXJpZXMsIG1lc3NhZ2U9RkFMU0UsIGVjaG89Rn0KIyBsb2FkIGxpYnJhcmllcwpsaWJyYXJ5KHRpZHl2ZXJzZSkKbGlicmFyeShoZXJlKQpsaWJyYXJ5KGphbml0b3IpCmxpYnJhcnkoYnJvb20pCmxpYnJhcnkoYWZleCkKbGlicmFyeShlbW1lYW5zKQpsaWJyYXJ5KGtuaXRyKQpsaWJyYXJ5KGthYmxlRXh0cmEpCmxpYnJhcnkoZ2dzY2kpCmxpYnJhcnkocGF0Y2h3b3JrKQpsaWJyYXJ5KHNraW1yKQojIGluc3RhbGwucGFja2FnZXMoImRldnRvb2xzIikKIyBkZXZ0b29sczo6aW5zdGFsbF9naXRodWIoImVhc3lzdGF0cy9jb3JyZWxhdGlvbiIpCmxpYnJhcnkoImNvcnJlbGF0aW9uIikKb3B0aW9ucyhzY2lwZW49OTk5KSAjIHR1cm4gb2ZmIHNjaWVudGlmaWMgbm90YXRpb25zCm9wdGlvbnMoY29udHJhc3RzID0gYygnY29udHIuc3VtJywnY29udHIucG9seScpKSAjIHNldCB0aGUgY29udHJhc3Qgc3VtIGdsb2JhbGx5IApvcHRpb25zKGtuaXRyLmthYmxlLk5BID0gJycpCmBgYAoKCiMgUmVzZWFyY2ggUXVlc3Rpb24KCgpUaGUgYWltIG9mIHRoZSBzdHVkeSBpcyB0byB0ZXN0IGlmIHNpbXBsZSBhcmd1bWVudHMgYXJlIG1vcmUgZWZmZWN0aXZlIGluIGJlbGllZiByZXZpc2lvbiB0aGFuIG1vcmUgY29tcGxleCBhcmd1bWVudHMuIFRvIHRoYXQgZW5kLCB3ZSBwcmVzZW50IHBhcnRpY2lwYW50cyB3aXRoIGFuIGltYWdpbmFyeSBzY2VuYXJpbyAodHdvIGFsaWVuIGNyZWF0dXJlcyBvbiBhIHBsYW5ldCkgYW5kIGEgdGhlb3J5IChvbmUgY3JlYXR1cmUgaXMgcHJlZGF0b3IgYW5kIHRoZSBvdGhlciBvbmUgaXMgcHJleSkgYW5kIHdlIGFzayB0aGVtIHRvIHJhdGUgdGhlIGxpa2VsaWhvb2QgdHJ1dGggb2YgdGhlIHRoZW9yeSBiYXNlZCBvbiBhIHNpbXBsZSBmYWN0IChXZSBhZGFwdGVkIHRoaXMgbWV0aG9kIGZyb20gR3JlZ2cgZXQgYWwuLDIwMTc7IHNlZSB0aGUgb3JpZ2luYWwgc3R1ZHkgW2hlcmVdKGh0dHBzOi8vam91cm5hbHMuc2FnZXB1Yi5jb20vZG9pLzEwLjEwODAvMTc0NzAyMTguMjAxNS4xMDk5MTYyKSkuIFRoZW4sIGluIGEgYmV0d2Vlbi1zdWJqZWN0IG1hbmlwdWxhdGlvbiwgcGFydGljaXBhbnRzIHdpbGwgYmUgcHJlc2VudGVkIHdpdGggZWl0aGVyIDYgc2ltcGxlIGFyZ3VtZW50cyAoTW9kdXMgUG9uZW5zIGNvbmRpdGlvbmFscykgb3IgNiBtb3JlIGNvbXBsZXggYXJndW1lbnRzIChNb2R1cyBUb2xsZW5zIGNvbmRpdGlvbmFscyksIGFuZCB0aGV5IHdpbGwgYmUgYXNrZWQgdG8gcmF0ZSB0aGUgbGlrZWxpaG9vZCB0cnV0aCBvZiB0aGUgaW5pdGlhbCB0aGVvcnkgb24gNyBzdGFnZXMuIAoKVGhlIGZpcnN0IHN0YWdlIGlzIHRoZSBiYXNlIHJhdGluZyBzdGFnZS4gVGhlIG5leHQgdGhyZWUgc3RhZ2VzIGluY2x1ZGUgc3VwcG9ydGl2ZSBhcmd1bWVudHMgb2YgdGhlIHRoZW9yeSBhbmQgdGhlIGxhc3QgdGhyZWUgYXJndW1lbnRzIGluY2x1ZGUgZGlzcHJvdmluZyBhcmd1bWVudHMgb2YgdGhlIHRoZW9yeS4gV2UgaHlwb3RoZXNpemVkIHRoYXQgdGhlIGdyb3VwIHdpdGggc2ltcGxlIGFyZ3VtZW50cyBzaG93cyBiZXR0ZXIgcGVyc3Vhc2lvbiAoYXMgaXQgcmVmbGVjdHMgaW4gaGlnaGVyIHJhdGluZ3MgZm9yIHRoZSBzdXBwb3J0aXZlIGFyZ3VtZW50cykgYW5kIGJldHRlciBkaXNzdWFzaW9uIChhcyBpdCByZWZsZWN0cyBpbiBsb3dlciByYXRpbmdzIGZvciB0aGUgb3Bwb3NpbmcgYXJndW1lbnRzKS4KCkluIHRoZSBsYXN0IHBhcnQgb2YgdGhlIHN0dWR5LCBwYXJ0aWNpcGFudHMgd2lsbCBiZSBhc2tlZCB0byBhbnN3ZXIgc2V2ZXJhbCBjb2duaXRpdmUgY2FwYWNpdHkvc3R5bGUgbWVhc3VyZXMgaW5jbHVkaW5nIENSVCwgQU9ULUUsIG1pbmR3YXJlLCBhbmQgbnVtZXJhY3kgc2NhbGVzLiBXZSBoeXBvdGhlc2l6ZWQgdGhhdCBjb2duaXRpdmUgYWJpbGl0eSwgY29nbml0aXZlIHN0eWxlLCBhbmQgb3Blbi1taW5kZWRuZXNzIGFyZSBwb3NpdGl2ZSBwcmVkaWN0b3JzIG9mIHBlcnN1YXNpb24gYW5kIGRpc3N1YXNpb24uIFRoZXNlIGFzc29jaWF0aW9ucyBzaG91bGQgYmUgbW9yZSBwcm9ub3VuY2VkIGZvciBwYXJ0aWNpcGFudHMgaW4gdGhlIGdyb3VwIHdpdGggY29tcGxleCBhcmd1bWVudHMgYmVjYXVzZSB0aGUgYWJpbGl0eSBhbmQgd2lsbGluZ25lc3MgdG8gZW5nYWdlIGluIGRlbGliZXJhdGl2ZSB0aGlua2luZyBtYXkgZmF2b3IgcGFydGljaXBhbnRzIHRvIGFzc2VzcyB0aGUgdW5kZXJseWluZyBsb2dpY2FsIHN0cnVjdHVyZSBvZiB0aG9zZSBhcmd1bWVudHMuIEhvd2V2ZXIsIGZvciBwYXJ0aWNpcGFudHMgaW4gdGhlIHNpbXBsZSBncm91cCwgdGhlIGxvZ2ljYWwgc3RydWN0dXJlIG9mIGFyZ3VtZW50cyBpcyBtb3JlIGV2aWRlbnQsIHNvIHBhcnRpY2lwYW50cyB3aXRoIGxvd2VyIGFiaWxpdHkgY2FuIHN0aWxsIGFzc2VzcyB0aGUgbG9naWNhbCBzdGF0dXMgb2YgdGhvc2UgYXJndW1lbnRzLgogCgpgYGB7ciBmaWcuYWxpZ249J2NlbnRlcicsIGVjaG89RkFMU0V9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKGhlcmUoJ2lucHV0cycsJ2V4cF9kZXNpZ24ucG5nJykpCmBgYAoKVGh1cywgb3VyIGh5cG90aGVzZXMgZm9yIHRoaXMgZXhwZXJpbWVudCBhcmUgYXMgZm9sbG93czoKCi0gUGFydGljaXBhbnRzIGluIHRoZSBncm91cCB3aXRoIHNpbXBsZSBhcmd1bWVudHMgaGF2ZSBoaWdoZXIgcmF0aW5ncyBmb3Igc3VwcG9ydGl2ZSBhcmd1bWVudHMgKFRoZXkgYXJlIG1vcmUgZWFzaWx5IHBlcnN1YWRlZCB0aGFuIHRob3NlIGluIHRoZSBncm91cCB3aXRoIGNvbXBsZXggYXJndW1lbnRzKS4KCi0gUGFydGljaXBhbnRzIGluIHRoZSBncm91cCB3aXRoIHNpbXBsZSBhcmd1bWVudHMgaGF2ZSBsb3dlciByYXRpbmdzIGZvciBvcHBvc2luZyBhcmd1bWVudHMgKFRoZXkgYXJlIG1vcmUgZWFzaWx5IGRpc3N1YWRlZCB0aGFuIHRob3NlIGluIHRoZSBncm91cCB3aXRoIGNvbXBsZXggYXJndW1lbnRzKS4KCi0gVGhlcmUgYXJlIHNpZ25pZmljYW50IGFzc29jaWF0aW9ucyBiZXR3ZWVuIENSVCwgQU9ULUUsIE51bWVyYWN5LCBhbmQgbWluZHdhcmUgd2l0aCBib3RoIHBlcnN1YXNpb24gYW5kIGRpc3N1YXNpb24gaW5kZXhlcyBpbiBlYWNoIGdyb3VwIGFuZCBpbiB0aGUgZW50aXJlIHNhbXBsZS4gVGhlIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIHRoZXNlIG1lYXN1cmVzIHNob3VsZCBiZSBzdHJvbmdlciwgYWx0aG91Z2ggbm90IHNpZ25pZmljYW50bHksIGZvciBwYXJ0aWNpcGFudHMgaW4gdGhlIGdyb3VwIHdpdGggY29tcGxleCBhcmd1bWVudHMuCgoKYGBge3IgZWNobz1GQUxTRSwgb3V0LndpZHRoPSI1NTBweCIsIG91dC5oZWlnaHQ9IjQwMHB4In0Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoaGVyZSgnaW5wdXRzJywncHJlZGljdGlvbl9wbG90LnBuZycpKQpgYGAKCgojIEdldHRpbmcgUmVhZHkKCkZpcnN0LCB3ZSBuZWVkIHRvIGRlc2lnbiB0aGUgZXhwZXJpbWVudC4gRm9yIHRoaXMgZXhwZXJpbWVudCwgd2UgdXNlIG9ubGluZSBwbGF0Zm9ybXMgZm9yIGRhdGEgY29sbGVjdGlvbi4gVGhlcmUgYXJlIHNldmVyYWwgb3B0aW9ucyBzdWNoIGFzIEdvcmlsbGEsIEpTcHN5Y2gsIFF1YWx0cmljcywgcHN5Y2hvSlMgKHBhdmxvdmlhKSwgZXRjLiBTaW5jZSB3ZSBkbyBub3QgbmVlZCBhbnkgcmVhY3Rpb24gdGltZSBkYXRhLCB3ZSBzaW1wbHkgdXNlIFF1YWx0cmljcy4gRm9yIGFuIG92ZXJ2aWV3IG9mIGRpZmZlcmVudCBsYWItYmFzZWQgYW5kIG9ubGluZSBwbGF0Zm9ybXMsIHNlZSBbaGVyZV0oaHR0cHM6Ly9vbWlkZ2hhc2VtaTIxLmdpdGh1Yi5pby9odW1hbl9kYXRhL1NjcmlwdHMvYmVoYXZpb3JhbF9kYXRhLmh0bWwpLiAKCk5leHQsIHdlIG5lZWQgdG8gZGVjaWRlIG9uIHRoZSBudW1iZXIgb2YgcGFydGljaXBhbnRzIChzYW1wbGUgc2l6ZSkuIEZvciB0aGlzIHN0dWR5LCB3ZSBkbyBub3Qgc3VlIHBvd2VyIGFuYWx5c2lzIHNpbmNlIHdlIGNhbm5vdCBhY2Nlc3MgbW9yZSB0aGFuIDEyMCBwYXJ0aWNpcGFudHMuIEhvd2V2ZXIsIGl0IGlzIGhpZ2hseSBzdWdnZXN0ZWQgY2FsY3VsYXRlIHNhbXBsZSBzaXplIHVzaW5nIHBvd2VyIGVzdGltYXRpb24uIFlvdSBjYW4gZmluZCBzb21lIG5pY2UgdHV0b3JpYWxzIG9uIGhvdyB0byBkbyB0aGF0IFtoZXJlXShodHRwczovL2p1bGlhbnF1YW5kdC5jb20vcG9zdC9wb3dlci1hbmFseXNpcy1ieS1kYXRhLXNpbXVsYXRpb24taW4tci1wYXJ0LWkvKSwgW2hlcmVdKGh0dHBzOi8vbmlja2NoLWsuZ2l0aHViLmlvL0Vjb25vbWV0cmljc1NsaWRlcy9XZWVrXzA4L1Bvd2VyX1NpbXVsYXRpb25zLmh0bWwpLCBhbmQgW2hlcmVdKGh0dHBzOi8vY3Jhbi5yLXByb2plY3Qub3JnL3dlYi9wYWNrYWdlcy9wYXJhbXRlc3QvdmlnbmV0dGVzL1NpbXVsYXRpbmctUG93ZXIuaHRtbCkuCgpBZnRlciB3ZSBjcmVhdGVkIHRoZSBleHBlcmltZW50IGFuZCBkZWNpZGVkIG9uIHRoZSBzYW1wbGUgc2l6ZSwgdGhlIG5leHQgc3RlcCBpcyB0byBwcmVyZXNpZ3RlciB0aGUgc3R1ZHkuIEhvd2V2ZXIsIGl0IHdvdWxkIGJlIGJldHRlciB0byBkbyBhIHBpbG90IHdpdGggNCBvciA1IHBhcnRpY2lwYW50cywgY2xlYW4gYWxsIHRoZSBkYXRhLCBkbyB0aGUgZGVzaXJlZCBhbmFseXNpcywgYW5kIHRoZW4gcHJlLXJlZ2lzdGVyIHRoZSBhbmFseXNpcyBhbmQgdGhvc2UgY29kZXMuIFlvdSBjYW4gZmluZCB0aGUgcHJlcmVnaXN0cmF0aW9uIGZvcm0gZm9yIHRoZSBjdXJyZW50IHN0dWR5IFtoZXJlXShodHRwczovL29zZi5pby83OXI2ZSkuCgpGaW5hbGx5LCB3ZSBuZWVkIHRvIHJlc3RydWN0dXJlIG91ciBwcm9qZWN0IGluIGEgdGlkeSBmb2xkZXIgd2l0aCBkaWZmZXJlbnQgc3ViLWZvbGRlcnMuIEhhdmluZyBhIGNsZWFuIGFuZCB0aWR5IGZvbGRlciBzdHJ1Y3R1cmUgY2FuIHNhdmUgdXMhIFRoZXJlIGFyZSBkaWZmZXJlbnQgZm9ybWF0cyBvZiBmb2xkZXIgc3RydWN0dXJlIChmb3IgZXhhbXBsZSwgc2VlIFtoZXJlXShodHRwOi8vbmlrb2xhLm1lL2ZvbGRlcl9zdHJ1Y3R1cmUuaHRtbCkgYW5kIFtoZXJlXShodHRwczovL3NsaWRlcy5jb20vZGpuYXZhcnJvL3dvcmtmbG93KSksIGJ1dCBmb3Igbm93LCB3ZSB1c2UgdGhlIGZvbGxvd2luZyBzdHJ1Y3R1cmU6CgpgYGB7ciBlY2hvPUZBTFNFLCBvdXQud2lkdGg9IjcwMHB4Iiwgb3V0LmhlaWdodD0iMjAwcHgifQprbml0cjo6aW5jbHVkZV9ncmFwaGljcyhoZXJlKCdpbnB1dHMnLCdmb2xkZXJfc3RydWN0dXJlLnBuZycpKQpgYGAKCgojIEludHJvZHVjdGlvbiB0byBSCmBgYHtyIG1lc3NhZ2U9RkFMU0UsIGV2YWw9Rn0KIyBsb2FkIGxpYnJhcmllcwpsaWJyYXJ5KHRpZHl2ZXJzZSkKbGlicmFyeShoZXJlKQpsaWJyYXJ5KGphbml0b3IpCmxpYnJhcnkoYnJvb20pCmxpYnJhcnkoYWZleCkKbGlicmFyeShlbW1lYW5zKQpsaWJyYXJ5KGtuaXRyKQpsaWJyYXJ5KGthYmxlRXh0cmEpCmxpYnJhcnkoZ2dzY2kpCmxpYnJhcnkocGF0Y2h3b3JrKQpsaWJyYXJ5KHNraW1yKQojIGluc3RhbGwucGFja2FnZXMoImRldnRvb2xzIikKIyBkZXZ0b29sczo6aW5zdGFsbF9naXRodWIoImVhc3lzdGF0cy9jb3JyZWxhdGlvbiIpCmxpYnJhcnkoImNvcnJlbGF0aW9uIikKb3B0aW9ucyhzY2lwZW49OTk5KSAjIHR1cm4gb2ZmIHNjaWVudGlmaWMgbm90YXRpb25zCm9wdGlvbnMoY29udHJhc3RzID0gYygnY29udHIuc3VtJywnY29udHIucG9seScpKSAjIHNldCB0aGUgY29udHJhc3Qgc3VtIGdsb2JhbGx5IApvcHRpb25zKGtuaXRyLmthYmxlLk5BID0gJycpCmBgYAoKUiBjYW4gYmUgdXNlZCBhcyBhIGNhbGN1bGF0b3IuIEZvciBtYXRoZW1hdGljYWwgcHVycG9zZXMsIGJlIGNhcmVmdWwgb2YgdGhlIG9yZGVyIGluIHdoaWNoIFIgZXhlY3V0ZXMgdGhlIGNvbW1hbmRzLgoKYGBge3J9CjEwICsgMTAKCjQgXiAyCgooMjUwIC8gNTAwKSAqIDEwMApgYGAKClIgaXMgYSBiaXQgZmxleGlibGUgd2l0aCBzcGFjaW5nIChidXQgbm8gc3BhY2luZyBpbiB0aGUgbmFtZSBvZiB2YXJpYWJsZXMgYW5kIHdvcmRzKQoKYGBge3J9CjEwKzEwCgoxMCAgICAgICAgICAgICAgICAgKyAgICAgICAgICAgMTAKYGBgCgpSIGNhbiBzb21ldGltZXMgdGVsbCB0aGF0IHlvdSdyZSBub3QgZmluaXNoZWQgeWV0CgpgYGB7ciBldmFsPUZ9CjEwICsKYGBgCgpIb3cgdG8gY3JlYXRlIGEgKnZhcmlhYmxlKj8gVmFyaWFibGUgYXNzaWdubWVudCB1c2luZyBgPC1gIGFuZCBgPWAuIE5vdGUgdGhhdCBSIGlzIGNhc2Ugc2Vuc2l0aXZlIGZvciBldmVyeXRoaW5nCgpgYGB7cn0KcGF5IDwtIDI1MAoKbW9udGggPSAxMgoKcGF5ICogbW9udGgKCnNhbGFyeSA8LSBwYXkgKiBtb250aApgYGAKCgpGZXcgcG9pbnRzIGluIG5hbWluZyB2YXJpYWJsZXMgYW5kIHZlY3RvcnM6IHVzZSBzaG9ydCwgaW5mb3JtYXRpdmUgd29yZHMsIGtlZXAgc2FtZSBtZXRob2QgKGUuZy4sIG5vdCB1c2luZyBjYXBpdGFsIHdvcmRzLCB1c2Ugb25seSBfIG9yIC4gKS4KCiMjIEZ1bmN0aW9uIApGdW5jdGlvbiBpcyBhIHNldCBvZiBzdGF0ZW1lbnRzIGNvbWJpbmVkIHRvZ2V0aGVyIHRvIHBlcmZvcm0gYSBzcGVjaWZpYyB0YXNrLiBXaGVuIHdlIHVzZSBhIGJsb2NrIG9mIGNvZGUgcmVwZWF0ZWRseSwgd2UgY2FuIGNvbnZlcnQgaXQgdG8gYSBmdW5jdGlvbi4gVG8gd3JpdGUgYSBmdW5jdGlvbiwgZmlyc3QsIHlvdSBuZWVkIHRvICpkZWZpbmUqIGl0OgoKYGBge3J9Cm15X211bHRpcGxpZXIgPC0gZnVuY3Rpb24oYSxiKXsKICByZXN1bHQgPSBhICogYgogIHJldHVybiAocmVzdWx0KQp9CmBgYAoKVGhpcyBjb2RlIGRvIG5vdGhpbmcuIFRvIGdldCBhIHJlc3VsdCwgeW91IG5lZWQgdG8gKmNhbGwqIGl0OgoKYGBge3J9Cm15X211bHRpcGxpZXIgKDIsNCkKYGBgCgpGb3J0dW5hdGVseSwgeW91IGRvIG5vdCBuZWVkIHRvIHdyaXRlIGV2ZXJ5dGhpbmcgZnJvbSBzY3JhdGNoLiBSIGhhcyBsb3RzIG9mIGJ1aWx0LWluIGZ1bmN0aW9ucyB0aGF0IHlvdSBjYW4gdXNlOgpgYGB7cn0Kcm91bmQoNTQuNjc4NykKcm91bmQoNTQuNTc4NywgZGlnaXRzID0gMikKYGBgCgpVc2UgYD9gIGJlZm9yZSB0aGUgZnVuY3Rpb24gbmFtZSB0byBnZXQgc29tZSBoZWxwLiBGb3IgZXhhbXBsZSwgYD9yb3VuZGAuIFlvdSB3aWxsIHNlZSBtYW55IGZ1bmN0aW9ucyBpbiB0aGUgcmVzdCBvZiB0aGUgd29ya3Nob3AuCgojIyBCYXNpYyBEYXRhIFR5cGVzIGluIFI6CgpmdW5jdGlvbiBgY2xhc3MoKWAgaXMgdXNlZCB0byBzaG93IHdoYXQgaXMgdGhlIHR5cGUgb2YgYSB2YXJpYWJsZS4KCgoxLiAqTG9naWNhbCo6IGBUUlVFYCwgYEZBTFNFYCBjYW4gYmUgYWJicmV2aWF0ZWQgYXMgYFRgLCBgRmAuICBUaGV5IGhhcyB0byBiZSBjYXBpdGFsLCAndHJ1ZScgaXMgbm90IGEgbG9naWNhbCBkYXRhOgpgYGB7cn0KY2xhc3MoVFJVRSkKY2xhc3MoRikKYGBgCgoyLiAqTnVtZXJpYyo6IGFsbCBudW1iZXJzIGUuZy4gNSwgIDEwLjUsICAxMSwzNzsgIGEgc3BlY2lhbCB0eXBlIG9mIG51bWVyaWMgaXMgImludGVnZXIiIHdoaWNoIGlzIG51bWJlcnMgd2l0aG91dCBkZWNpbWFsLiBJbnRlZ2VycyBhcmUgYWx3YXlzIG51bWVyaWMsIGJ1dCBudW1lcmljIGlzIG5vdCBhbHdheXMgaW50ZWdlcjoKYGBge3J9CmNsYXNzKDIpCmNsYXNzKDEzLjQ2KQpgYGAKCjMuICpDaGFyYWN0ZXIqOiB0ZXh0IGZvciBleGFtcGxlLCAiSSBsb3ZlIFIiIG9yICI0IiBvciAiNC41IjoKYGBge3J9CmNsYXNzKCJoYSBoYSBoYSBoYSIpCmNsYXNzKCI1Ni42IikKY2xhc3MoIlRSVUUiKQpgYGAKCkNhbiB3ZSBjaGFuZ2UgdGhlIHR5cGUgb2YgZGF0YSBpbiBhIHZhcmlhYmxlPyBZZXMsIHlvdSBuZWVkIHRvIHVzZSB0aGUgZnVuY3Rpb24gYGFzLi0tLSgpYAoKYGBge3J9CmFzLm51bWVyaWMoVFJVRSkKYXMuY2hhcmFjdGVyKDQpCmFzLm51bWVyaWMoIjQuNSIpCmFzLm51bWVyaWMoIkhlbGxvIikKYGBgCgoKIyMgRGF0YSBTdHJ1Y3R1cmVzIGluIFIKCgoqKlZlY3RvcioqOiB3aGVuIHRoZXJlIGFyZSBtb3JlIHRoYW4gb25lIG51bWJlciBvciBsZXR0ZXIgc3RvcmVkLiBVc2UgdGhlIGNvbWJpbmUgZnVuY3Rpb24gYygpIGZvciB0aGF0LgoKYGBge3J9CnNhbGUgPC0gYygxLCAyLCAzLDQsIDUsIDYsIDcsIDgsIDksIDEwKSAjIGFsc28gc2FsZSA8LSBjKDE6MTApCgpzYWxlIDwtIGMoMToxMCkKCnNhbGUgKiBzYWxlCmBgYAoKKlN1YnNldHRpbmcgYSB2ZWN0b3IqOgoKYGBge3J9CmRheXMgPC0gYygiU2F0dXJkYXkiLCAiU3VuZGF5IiwgIk1vbmRheSIsICJUdWVzZGF5IiwgIldlZG5lc2RheSIsICJUaHVyc2RheSIsICJGcmlkYXkiKQoKZGF5c1syXQpkYXlzWy0yXQoKZGF5c1tjKDIsIDMsIDQpXQpgYGAKCgojIyMgRXhlcmNpc2UKCkNyZWF0ZSBhIHZlY3RvciBuYW1lZCBgbXlfdmVjdG9yYCB3aXRoIG51bWJlcnMgZnJvbSAwIHRvIDEwMDAgaW4gaXQ6CgpgYGB7cn0KbXlfdmVjdG9yIDwtICgwOjEwMDApCgptZWFuKG15X3ZlY3RvcikKbWVkaWFuKG15X3ZlY3RvcikKbWluKG15X3ZlY3RvcikKcmFuZ2UobXlfdmVjdG9yKQpjbGFzcyhteV92ZWN0b3IpCnN1bShteV92ZWN0b3IpCnNkKG15X3ZlY3RvcikKYGBgCgoqKkxpc3QqKjogYWxsb3dzIHlvdSB0byBnYXRoZXIgYSB2YXJpZXR5IG9mIG9iamVjdHMgdW5kZXIgb25lIG5hbWUgKHRoYXQgaXMsIHRoZSBuYW1lIG9mIHRoZSBsaXN0KSBpbiBhbiBvcmRlcmVkIHdheS4gVGhlc2Ugb2JqZWN0cyBjYW4gYmUgbWF0cmljZXMsIHZlY3RvcnMsIGRhdGEgZnJhbWVzLCBldmVuIG90aGVyIGxpc3QuCgpgYGB7cn0KbXlfbGlzdCA9IGxpc3Qoc2FsZSwgMSwgMywgNDo3LCAiSEVMTE8iLCAiaGVsbG8iLCBGQUxTRSkKbXlfbGlzdApgYGAKCioqRmFjdG9yKio6IEZhY3RvcnMgc3RvcmUgdGhlIHZlY3RvciBhbG9uZyB3aXRoIHRoZSBkaXN0aW5jdCB2YWx1ZXMgb2YgdGhlIGVsZW1lbnRzIGluIHRoZSB2ZWN0b3IgYXMgbGFiZWxzLiBUaGUgbGFiZWxzIGFyZSBhbHdheXMgY2hhcmFjdGVyIGlycmVzcGVjdGl2ZSBvZiB3aGV0aGVyIGl0IGlzIG51bWVyaWMgb3IgY2hhcmFjdGVyLiBGb3IgZXhhbXBsZSwgdmFyaWFibGUgZ2VuZGVyIHdpdGggIm1hbGUiIGFuZCAiZmVtYWxlIiBlbnRyaWVzOgoKYGBge3J9CmdlbmRlciA8LSBjKCJtYWxlIiwgIm1hbGUiLCAibWFsZSIsICIgZmVtYWxlIiwgImZlbWFsZSIsICJmZW1hbGUiKQpnZW5kZXIgPC0gZmFjdG9yKGdlbmRlcikKYGBgCgpSIG5vdyB0cmVhdHMgZ2VuZGVyIGFzIGEgbm9taW5hbCAoY2F0ZWdvcmljYWwpIHZhcmlhYmxlOiAxPWZlbWFsZSwgMj1tYWxlIGludGVybmFsbHkgKGFscGhhYmV0aWNhbGx5KS4KYGBge3J9CnN1bW1hcnkoZ2VuZGVyKQpgYGAKCipRdWVzdGlvbio6IHdoeSB3aGVuIHdlIHJhbiB0aGUgYWJvdmUgZnVuY3Rpb24gaS5lLiBzdW1tYXJ5KCksIGl0IHNob3dlZCB0aHJlZSBhbmQgbm90IHR3byBsZXZlbHMgb2YgdGhlIGRhdGE/ICpIaW50KjogcnVuICdnZW5kZXInLgoKYGBge3J9CmdlbmRlcgpgYGAKClNvLCBiZSBjYXJlZnVsIG9mIHNwYWNlcyEKCiMjIyBFeGVyY2lzZQpDcmVhdGUgYSBnZW5kZXIgZmFjdG9yIHdpdGggMzAgbWFsZSBhbmQgNDAgZmVtYWxlcyAoKkhpbnQqOiB1c2UgdGhlIGByZXAoKWAgZnVuY3Rpb24pOgpgYGB7cn0KZ2VuZGVyIDwtIGMocmVwKCJtYWxlIiwzMCksIHJlcCgiZmVtYWxlIiwgNDApKQpnZW5kZXIgPC0gZmFjdG9yKGdlbmRlcikKZ2VuZGVyCmBgYAoKVGhlcmUgYXJlIHR3byB0eXBlcyBvZiBjYXRlZ29yaWNhbCB2YXJpYWJsZXM6IG5vbWluYWwgYW5kIG9yZGluYWwuIEhvdyB0byBjcmVhdGUgb3JkZXJlZCBmYWN0b3JzICh3aGVuIHRoZSB2YXJpYWJsZSBpcyBub21pbmFsIGFuZCB2YWx1ZXMgY2FuIGJlIG9yZGVyZWQpPyBXZSBzaG91bGQgYWRkIHR3byBhZGRpdGlvbmFsIGFyZ3VtZW50cyB0byB0aGUgYGZhY3RvcigpYCBmdW5jdGlvbjogYG9yZGVyZWQgPSBUUlVFYCwgYW5kIGBsZXZlbHMgPSBjKCJsZXZlbDEiLCAibGV2ZWwyIilgLiBGb3IgZXhhbXBsZSwgd2UgaGF2ZSBhIHZlY3RvciB0aGF0IHNob3dzIHBhcnRpY2lwYW50cycgZWR1Y2F0aW9uIGxldmVsLgoKYGBge3J9CmVkdTwtYygzLDIsMyw0LDEsMiwyLDMsNCkKCmVkdWNhdGlvbjwtZmFjdG9yKGVkdSwgb3JkZXJlZCA9IFRSVUUpCmxldmVscyhlZHVjYXRpb24pIDwtIGMoIlByaW1hcnkgc2Nob29sIiwiaGlnaCBzY2hvb2wiLCJDb2xsZWdlIiwiVW5pIGdyYWR1YXRlZCIpCmVkdWNhdGlvbgpgYGAKCiMjIyBFeGVyY2lzZQpXZSBoYXZlIGEgZmFjdG9yIHdpdGggYHBhdGllbnRgIGFuZCBgY29udHJvbGAgdmFsdWVzLiBIZXJlLCB0aGUgZmlyc3QgbGV2ZWwgaXMgY29udHJvbCBhbmQgdGhlIHNlY29uZCBsZXZlbCBpcyBwYXRpZW50LiBDaGFuZ2UgdGhlIG9yZGVyIG9mIGxldmVscywgc28gcGF0aWVudCB3b3VsZCBiZSB0aGUgZmlyc3QgbGV2ZWw6CgpgYGB7cn0KaGVhbHRoX3N0YXR1cyA8LSBmYWN0b3IoYyhyZXAoJ3BhdGllbnQnLDUpLHJlcCgnY29udHJvbCcsNSkpKQpoZWFsdGhfc3RhdHVzCgpoZWFsdGhfc3RhdHVzX3Jlb3JkZXJlZCA8LSBmYWN0b3IoaGVhbHRoX3N0YXR1cywgbGV2ZWxzID0gYygncGF0aWVudCcsJ2NvbnRyb2wnKSkKaGVhbHRoX3N0YXR1c19yZW9yZGVyZWQKYGBgCgpGaW5hbGx5LCBjYW4geW91IHJlbGFiZWwgYm90aCBsZXZlbHMgdG8gdXBwZXJjYXNlIGNoYXJhY3RlcnM/ICgqSGludCo6IGNoZWNrIGA/ZmFjdG9yYCkKCmBgYHtyfQpoZWFsdGhfc3RhdHVzX3JlbGFiZWxlZCA8LSBmYWN0b3IoaGVhbHRoX3N0YXR1cywgbGV2ZWxzID0gYygncGF0aWVudCcsJ2NvbnRyb2wnKSwgbGFiZWxzID0gYygnUGF0aWVudCcsJ0NvbnRyb2wnKSkKaGVhbHRoX3N0YXR1c19yZWxhYmVsZWQKYGBgCgoKKipNYXRyaWNlcyoqOiBBbGwgY29sdW1ucyBpbiBhIG1hdHJpeCBtdXN0IGhhdmUgdGhlIHNhbWUgbW9kZShudW1lcmljLCBjaGFyYWN0ZXIsIGV0Yy4pIGFuZCB0aGUgc2FtZSBsZW5ndGguIEl0IGNhbiBiZSBjcmVhdGVkIHVzaW5nIGEgdmVjdG9yIGlucHV0IHRvIHRoZSBtYXRyaXggZnVuY3Rpb24uCgpgYGB7cn0KbXlfbWF0cml4ID0gbWF0cml4KGMoMSwyLDMsNCw1LDYsNyw4LDkpLCBucm93ID0gMywgbmNvbCA9IDMpCgpteV9tYXRyaXgKYGBgCgoqKkRhdGEgZnJhbWVzKio6ICh0d28tZGltZW5zaW9uYWwgb2JqZWN0cykgY2FuIGhvbGQgbnVtZXJpYywgY2hhcmFjdGVyIG9yIGxvZ2ljYWwgdmFsdWVzLiBXaXRoaW4gYSBjb2x1bW4gYWxsIGVsZW1lbnRzIGhhdmUgdGhlIHNhbWUgZGF0YSB0eXBlLCBidXQgZGlmZmVyZW50IGNvbHVtbnMgY2FuIGJlIG9mIGRpZmZlcmVudCBkYXRhIHR5cGUuIExldCdzIGNyZWF0ZSBhIGRhdGFmcmFtZToKCmBgYHtyfQppZCA8LSAxOjIwMApncm91cCA8LSBjKHJlcCgiUHN5Y2hvdGhlcmFweSIsIDEwMCksIHJlcCgiTWVkaWNhdGlvbiIsIDEwMCkpCnJlc3BvbnNlIDwtIGMocm5vcm0oMTAwLCBtZWFuID0gMzAsIHNkID0gNSksCiAgICAgICAgICAgICBybm9ybSgxMDAsIG1lYW4gPSAyNSwgc2QgPSA1KSkKCm15X2RhdGFmcmFtZSA8LWRhdGEuZnJhbWUoUGF0aWVudCA9IGlkLAogICAgICAgICAgICAgICAgICAgICAgICAgIFRyZWF0bWVudCA9IGdyb3VwLAogICAgICAgICAgICAgICAgICAgICAgICAgIFJlc3BvbnNlID0gcmVzcG9uc2UpCmBgYAoKV2UgYWxzbyBjb3VsZCBoYXZlIGRvbmUgdGhlIGJlbG93CgpgYGB7cn0KbXlfZGF0YWZyYW1lIDwtZGF0YS5mcmFtZShQYXRpZW50ID0gYygxOjIwMCksCiAgICAgICAgICAgICAgICAgICAgICAgICAgVHJlYXRtZW50ID0gYyhyZXAoIlBzeWNob3RoZXJhcHkiLCAxMDApLCByZXAoIk1lZGljYXRpb24iLCAxMDApKSwKICAgICAgICAgICAgICAgICAgICAgICAgICBSZXNwb25zZSA9IGMocm5vcm0oMTAwLCBtZWFuID0gMzAsIHNkID0gNSksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJub3JtKDEwMCwgbWVhbiA9IDI1LCBzZCA9IDUpKSkKYGBgCgpJbiBsYXJnZSBkYXRhIHNldHMsIHRoZSBmdW5jdGlvbiBoZWFkKCkgZW5hYmxlcyB5b3UgdG8gc2hvdyB0aGUgZmlyc3Qgb2JzZXJ2YXRpb25zIG9mIGEgZGF0YSBmcmFtZXMuIFNpbWlsYXJseSwgdGhlIGZ1bmN0aW9uIHRhaWwoKSBwcmludHMgb3V0IHRoZSBsYXN0IG9ic2VydmF0aW9ucyBpbiB5b3VyIGRhdGEgc2V0LgoKYGBge3IgZXZhbD1GfQpoZWFkKG15X2RhdGFmcmFtZSkgCnRhaWwobXlfZGF0YWZyYW1lKQpgYGAKCmBgYHtyIGVjaG89Rn0KaGVhZChteV9kYXRhZnJhbWUpICU+JQogIGtuaXRyOjprYWJsZSgpICU+JQogIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIiwgImJvcmRlcmVkIiwgImNvbmRlbnNlZCIpLCBmaXhlZF90aGVhZCA9IFQsIGZ1bGxfd2lkdGggPSBUKQp0YWlsKG15X2RhdGFmcmFtZSklPiUKICBrbml0cjo6a2FibGUoKSAlPiUKICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIsICJib3JkZXJlZCIsICJjb25kZW5zZWQiKSwgZml4ZWRfdGhlYWQgPSBULCBmdWxsX3dpZHRoID0gVCkKYGBgCgpTaW1pbGFyIHRvIHZlY3RvcnMgYW5kIG1hdHJpY2VzLCBicmFja2V0cyBbXSBhcmUgdXNlZCB0byBzZWxlY3RzIGRhdGEgZnJvbSByb3dzIGFuZCBjb2x1bW5zIGluIGRhdGEuZnJhbWVzOgoKYGBge3J9Cm15X2RhdGFmcmFtZVszNSwgM10KYGBgCgojIyMgRXhlcmNpc2UKCkhvdyBjYW4gd2UgZ2V0IGFsbCBjb2x1bW5zLCBidXQgb25seSBmb3IgdGhlIGZpcnN0IDEwIHBhcnRpY2lwYW50cz8KCmBgYHtyIGV2YWw9Rn0KbXlfZGF0YWZyYW1lWzE6MTAsIF0KYGBgCgpgYGB7ciBlY2hvPUZ9CmtuaXRyOjprYWJsZShteV9kYXRhZnJhbWVbMToxMCwgXSkgJT4lCiAga2FibGVfc3R5bGluZyhib290c3RyYXBfb3B0aW9ucyA9IGMoInN0cmlwZWQiLCAiYm9yZGVyZWQiLCAiY29uZGVuc2VkIiksIGZpeGVkX3RoZWFkID0gVCwgZnVsbF93aWR0aCA9IFQpCgpgYGAKSG93IHRvIGdldCBvbmx5IHRoZSBSZXNwb25zZSBjb2x1bW4gZm9yIGFsbCBwYXJ0aWNpcGFudHM/CgpgYGB7cn0KbXlfZGF0YWZyYW1lWyAsIDNdCmBgYAoKQW5vdGhlciBlYXNpZXIgd2F5IGZvciBzZWxlY3RpbmcgcGFydGljdWxhciBpdGVtcyBpcyB1c2luZyB0aGVpciBuYW1lcyB0aGF0IGlzIG1vcmUgaGVscGZ1bCB0aGFuIG51bWJlciBvZiB0aGUgcm93cyBpbiBsYXJnZSBkYXRhIHNldHM6CmBgYHtyIGV2YWw9Rn0KbXlfZGF0YWZyYW1lWyAsICJSZXNwb25zZSJdCiMgT1I6Cm15X2RhdGFmcmFtZSRSZXNwb25zZQoKYGBgCgoKIyBEYXRhIENsZWFuaW5nCgpOb3csIHN1cHBvc2Ugd2UgdGVzdGVkIDE0MSBzdHVkZW50cy4gRmlyc3QsIGxldCdzIHJlYWQgYW5kIGNoZWNrIHRoZSB1bmNsZWFuZWQgZGF0YToKYGBge3IgbWVzc2FnZT1GLCB3YXJuaW5nPUYsIGV2YWw9Rn0KIyByZWFkIHRoZSByYXcgZGF0YQpyYXdfZGF0YSA8LSByZWFkX2NzdihoZXJlKCJyYXdfZGF0YSIsInJhd19hcmd1bWVudGF0aXZlX2V4cDEuY3N2IikpCmhlYWQocmF3X2RhdGEpCmBgYAoKYGBge3IgbWVzc2FnZT1GLCB3YXJuaW5nPUYsIGVjaG89Rn0KIyByZWFkIHRoZSByYXcgZGF0YQpyYXdfZGF0YSA8LSByZWFkX2NzdihoZXJlKCJyYXdfZGF0YSIsInJhd19hcmd1bWVudGF0aXZlX2V4cDEuY3N2IikpCgprbml0cjo6a2FibGUoaGVhZChyYXdfZGF0YSkpICU+JQogIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIiwgImJvcmRlcmVkIiwgImNvbmRlbnNlZCIpLCBmaXhlZF90aGVhZCA9IFQsIGZ1bGxfd2lkdGggPSBGKSU+JQogIHNjcm9sbF9ib3god2lkdGggPSAiNzgwcHgiKQpgYGAKCk5vdywgbGV0J3MgZG8gc29tZSBjbGVhbmluaW5nIHVzaW5nIGBkcGx5cmAsIGB0aWR5cmAgYW5kIG90aGVyIGB0aWR5dmVyc2VgIGxpYnJhcmllcy4gRmluYWxseSwgd2Ugd2lsbCBjaGVjayB0aGUgZGF0YToKYGBge3IgbWVzc2FnZT1GLCB3YXJuaW5nPUYsIGV2YWw9Rn0KY2xlYW5lZF9kYXRhIDwtIHJhd19kYXRhICU+JSAKICBmaWx0ZXIocHJvZ3Jlc3MgPT0gMTAwKSAlPiUgIyBmaWx0ZXIgb3V0IHVuZmluaXNoZWQgcGFydGljaXBhbnRzCiAgc2VsZWN0KC1lbmRfZGF0ZSwgLXN0YXR1cywtaXBfYWRkcmVzcywgLWR1cmF0aW9uX2luX3NlY29uZHMsIC1yZWNvcmRlZF9kYXRlOi11c2VyX2xhbmd1YWdlKSAlPiUgI3JlbW92ZSBzb21lIHVzZWxlc3MgY29sdW1ucwogIG11dGF0ZShvcGVubWluZGVkX3RvdGFsPSBvcGVubWluZGVkMStvcGVubWluZGVkMitvcGVubWluZGVkMytvcGVubWluZGVkNCtvcGVubWluZGVkNStvcGVubWluZGVkNitvcGVubWluZGVkNytvcGVubWluZGVkOCkgJT4lIyBjcmVhdGUgYSB0b3RhbCBzY29yZSBmb3Igb3VyIHF1ZXN0aW9ubmFpcmUKICBtdXRhdGUodGhpbmtpbmcxPSBjYXNlX3doZW4odGhpbmtpbmcxPT0nNCd+IDEsVH4wKSwKICAgICAgICAgdGhpbmtpbmcyPSBjYXNlX3doZW4odGhpbmtpbmcyPT0nMTAnfiAxLFR+MCksCiAgICAgICAgIHRoaW5raW5nMz0gY2FzZV93aGVuKHRoaW5raW5nMz09JzM5J34gMSxUfjApLAogICAgICAgICB0aGlua2luZ190b3RhbD0gdGhpbmtpbmcxICsgdGhpbmtpbmcyICsgdGhpbmtpbmczKSAlPiUKICBzZWxlY3QoLXRoaW5raW5nMTotb3Blbm1pbmRlZDgpICU+JQogIHBpdm90X2xvbmdlcihjb2xzID0gYyhzdGFnZTFfc2ltcGxlOnN0YWdlN19zaW1wbGUsc3RhZ2UxX2NvbXBsZXg6c3RhZ2U3X2NvbXBsZXgpLG5hbWVzX3RvID0gJ3N0YWdlJyx2YWx1ZXNfdG8gPSAndHJ1dGhfZXN0aW1hdGUnKSAlPiUgIyBtYWtlIG91ciBkYXRhZnJhbWUgbG9uZwogICNwaXZvdF93aWRlcihuYW1lc19mcm9tID0gc3RhZ2UsIHZhbHVlc19mcm9tPSB0cnV0aF9lc3RpbWF0ZSkgIyB0aGlzIGNvZGUgY2hhbmdlIG91ciBkYXRhZnJhbWUgYmFjayB0byB3aWRlCiAgZmlsdGVyKCFpcy5uYSh0cnV0aF9lc3RpbWF0ZSkpICU+JSAjcmVtb3ZlIHJvd3Mgd2l0aCB0cnV0aF9lc3RpbWF0ZSA9PSBOQQogIG11dGF0ZShzdGFnZT0gZ3N1YigiXy4qIiwgIiIsIHN0YWdlKSkgJT4lCiAgcmVuYW1lKGNvbnNlbnQ9IGNvbnNlbnRfZm9ybSkgJT4lICMgcmVuYW1lIGEgY29sdW1uCiAgI211dGF0ZV9pZihpcy5jaGFyYWN0ZXIsIGZhY3RvcikgJT4lCiAgbXV0YXRlKHN1YmplY3Q9IGZhY3RvcihzdWJqZWN0KSwgIyBjb252ZXJ0IGFsbCBjaGFyYWN0ZXJzIHRvIGZhY3RvcgogICAgICAgICBncm91cCA9IGZhY3Rvcihncm91cCksCiAgICAgICAgIHN0YWdlID0gZmFjdG9yKHN0YWdlKSkKYGBgCgoKYGBge3IgbWVzc2FnZT1GLCB3YXJuaW5nPUYsIGVjaG89Rn0KY2xlYW5lZF9kYXRhIDwtIHJhd19kYXRhICU+JSAKICBmaWx0ZXIocHJvZ3Jlc3MgPT0gMTAwKSAlPiUgIyBmaWx0ZXIgb3V0IHVuZmluaXNoZWQgcGFydGljaXBhbnRzCiAgc2VsZWN0KC1lbmRfZGF0ZSwgLXN0YXR1cywtaXBfYWRkcmVzcywgLWR1cmF0aW9uX2luX3NlY29uZHMsIC1yZWNvcmRlZF9kYXRlOi11c2VyX2xhbmd1YWdlKSAlPiUgI3JlbW92ZSBzb21lIHVzZWxlc3MgY29sdW1ucwogIG11dGF0ZShvcGVubWluZGVkX3RvdGFsPSBvcGVubWluZGVkMStvcGVubWluZGVkMitvcGVubWluZGVkMytvcGVubWluZGVkNCtvcGVubWluZGVkNStvcGVubWluZGVkNitvcGVubWluZGVkNytvcGVubWluZGVkOCkgJT4lIyBjcmVhdGUgYSB0b3RhbCBzY29yZSBmb3Igb3VyIHF1ZXN0aW9ubmFpcmUKICBtdXRhdGUodGhpbmtpbmcxPSBjYXNlX3doZW4odGhpbmtpbmcxPT0nNCd+IDEsVH4wKSwKICAgICAgICAgdGhpbmtpbmcyPSBjYXNlX3doZW4odGhpbmtpbmcyPT0nMTAnfiAxLFR+MCksCiAgICAgICAgIHRoaW5raW5nMz0gY2FzZV93aGVuKHRoaW5raW5nMz09JzM5J34gMSxUfjApLAogICAgICAgICB0aGlua2luZ190b3RhbD0gdGhpbmtpbmcxICsgdGhpbmtpbmcyICsgdGhpbmtpbmczKSAlPiUKICBzZWxlY3QoLXRoaW5raW5nMTotb3Blbm1pbmRlZDgpICU+JQogIHBpdm90X2xvbmdlcihjb2xzID0gYyhzdGFnZTFfc2ltcGxlOnN0YWdlN19zaW1wbGUsc3RhZ2UxX2NvbXBsZXg6c3RhZ2U3X2NvbXBsZXgpLG5hbWVzX3RvID0gJ3N0YWdlJyx2YWx1ZXNfdG8gPSAndHJ1dGhfZXN0aW1hdGUnKSAlPiUgIyBtYWtlIG91ciBkYXRhZnJhbWUgbG9uZwogICNwaXZvdF93aWRlcihuYW1lc19mcm9tID0gc3RhZ2UsIHZhbHVlc19mcm9tPSB0cnV0aF9lc3RpbWF0ZSkgIyB0aGlzIGNvZGUgY2hhbmdlIG91ciBkYXRhZnJhbWUgYmFjayB0byB3aWRlCiAgZmlsdGVyKCFpcy5uYSh0cnV0aF9lc3RpbWF0ZSkpICU+JSAjcmVtb3ZlIHJvd3Mgd2l0aCB0cnV0aF9lc3RpbWF0ZSA9PSBOQQogIG11dGF0ZShzdGFnZT0gZ3N1YigiXy4qIiwgIiIsIHN0YWdlKSkgJT4lCiAgcmVuYW1lKGNvbnNlbnQ9IGNvbnNlbnRfZm9ybSkgJT4lICMgcmVuYW1lIGEgY29sdW1uCiAgI211dGF0ZV9pZihpcy5jaGFyYWN0ZXIsIGZhY3RvcikgJT4lCiAgbXV0YXRlKHN1YmplY3Q9IGZhY3RvcihzdWJqZWN0KSwgIyBjb252ZXJ0IGFsbCBjaGFyYWN0ZXJzIHRvIGZhY3RvcgogICAgICAgICBncm91cCA9IGZhY3Rvcihncm91cCksCiAgICAgICAgIHN0YWdlID0gZmFjdG9yKHN0YWdlKSkKCmtuaXRyOjprYWJsZShoZWFkKGNsZWFuZWRfZGF0YSkpICU+JQogIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIiwgImJvcmRlcmVkIiwgImNvbmRlbnNlZCIpLCBmaXhlZF90aGVhZCA9IFQsIGZ1bGxfd2lkdGggPSBGKSU+JQogIHNjcm9sbF9ib3god2lkdGggPSAiNzgwcHgiKQpgYGAKCk9rLCBub3cgdGhlIGRhdGEgaXMgY2xlYW4gYW5kIHRpZHkgd2hpY2ggbWVhbnM6Cgo+IDEuIEVhY2ggdmFyaWFibGUgZm9ybXMgYSBjb2x1bW4uCjIuIEVhY2ggb2JzZXJ2YXRpb24gZm9ybXMgYSByb3cuCjMuIEVhY2ggdHlwZSBvZiBvYnNlcnZhdGlvbmFsIHVuaXQgZm9ybXMgYSB0YWJsZSAoW1dpY2toYW1dKGh0dHBzOi8vdml0YS5oYWQuY28ubnovcGFwZXJzL3RpZHktZGF0YS5wZGYpLCAyMDE0KS4KCkNoZWNrIHRoZSBkYXRhZnJhbWUgYW5kIGFsbCB0aGUgZGF0YSB0eXBlczoKYGBge3J9CnN0cihjbGVhbmVkX2RhdGEpCmBgYAoKRmluYWxseSwgd2Ugc2F2ZSBvdXIgZGF0YSB0byB0aGUgYGNsZWFuZWRfZGF0YWAgZm9sZGVyLgoKYGBge3J9CndyaXRlX2NzdihjbGVhbmVkX2RhdGEsIGhlcmUoImNsZWFuZWRfZGF0YSIsImFyZ3VtZW50YXRpdmVfZXhwMS5jc3YiKSkKYGBgCgojIERlc2NyaXB0aXZlIFN0YXRpc3RpY3MKCj4gTm90ZTogQWxsIHRoZSBkYXRhIHRoYXQgd2UgdXNlIGhlcmUgaXMgbWFuaXB1bGF0ZWQgKGZhYnJpY2F0ZWQpIGZvciB0ZWFjaGluZyBwdXJwdXNlcy4gSW4gb3VyIHN0dWR5LCB3ZSBmYWlsZWQgdG8gZmluZCBzdWNoIGJlYXV0aWZ1bCBhbmQgaW50ZXJlc3RpbmcgcmVzdWx0cy4KCk5vdywgbGV0J3MgZG8gc29tZSBkZXNjcmlwdGl2ZSBzdGF0aXN0aWNzLiBGaXJzdCwgd2UgY2FuIG9wZW4gYSBuZXcgc2NyaXB0IGNhbGxlZCBgYW5hbHlzaXNfZXhwMS5yYCBhbmQgcmVhZCB0aGUgY2xlYW5lZCBkYXRhIGFnYWluLiAKCmBgYHtyIG1lc3NhZ2U9Riwgd2FybmluZz1GLH0KZGF0YV9leHAxIDwtIHJlYWRfY3N2KGhlcmUoImNsZWFuZWRfZGF0YSIsImFyZ3VtZW50YXRpdmVfZXhwMS5jc3YiKSkKYGBgCgpIb3cgbWFueSBwYXJ0aWNpcGFudHMgaW4gdG90YWw/CgpgYGB7ciBtZXNzYWdlPUYsIHdhcm5pbmc9RiwgZXZhbD1GfQpkYXRhX2V4cDEgJT4lIHN1bW1hcmlzZShuPSBuX2Rpc3RpbmN0KHN1YmplY3QpKQpgYGAKCgpgYGB7ciBtZXNzYWdlPUYsIHdhcm5pbmc9RiwgZWNobz1GfQpkYXRhX2V4cDEgJT4lIHN1bW1hcmlzZShuPSBuX2Rpc3RpbmN0KHN1YmplY3QpKSU+JQogIGtuaXRyOjprYWJsZSgpICU+JQogIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIiwgImJvcmRlcmVkIiwgImNvbmRlbnNlZCIpLCBmdWxsX3dpZHRoID0gRikKYGBgCgpob3cgbWFueSBwYXJ0aWNpcGFudHMgaW4gZWFjaCBncm91cD8KYGBge3IgbWVzc2FnZT1GLCB3YXJuaW5nPUYsIGV2YWw9Rn0KZGF0YV9leHAxICU+JSAKICBncm91cF9ieShzdWJqZWN0KSAlPiUgCiAgZmlsdGVyKHJvd19udW1iZXIoKT09MSkgJT4lIAogIHVuZ3JvdXAgKCkgJT4lIAogIGdyb3VwX2J5KGdyb3VwKSAlPiUgCiAgY291bnQoKSAKYGBgCgpgYGB7ciBtZXNzYWdlPUYsIHdhcm5pbmc9RiwgZWNobz1GfQpkYXRhX2V4cDEgJT4lIGdyb3VwX2J5KHN1YmplY3QpICU+JSBmaWx0ZXIocm93X251bWJlcigpPT0xKSAlPiUgdW5ncm91cCAoKSAlPiUgZ3JvdXBfYnkoZ3JvdXApICU+JSBjb3VudCgpICU+JQogIGtuaXRyOjprYWJsZSgpICU+JQogIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIiwgImJvcmRlcmVkIiwgImNvbmRlbnNlZCIpLCBmaXhlZF90aGVhZCA9IFQpCmBgYAoKRmluZCB0aGUgbWVhbiBhbmQgc2QgZm9yIG51bWVyaWMgdmFyaWFibGVzIHVzaW5nIGJhc2UgUiBgc3VtbWFyeWAgZnVuY3Rpb246CgpgYGB7cn0KZGF0YV9leHAxICU+JSAKICBncm91cF9ieShzdWJqZWN0KSAlPiUgCiAgZmlsdGVyKHJvd19udW1iZXIoKT09MSkgJT4lIAogIHVuZ3JvdXAgKCkgJT4lCiAgc3VtbWFyeSgpCmBgYAoKQWx0ZXJuYXRpdmVseSwgd2UgY2FuIHVzZSBgYmFzZSBSIGBzdW1tYXJ5YCBmdW5jdGlvbmBza2ltcmAgbGlicmFyeToKYGBge3IgZXZhbD1GfQpkYXRhX2V4cDEgJT4lIAogIGdyb3VwX2J5KHN1YmplY3QpICU+JSAKICBmaWx0ZXIocm93X251bWJlcigpPT0xKSAlPiUgCiAgdW5ncm91cCAoKSAlPiUgCiAgZHBseXI6OnNlbGVjdCAoYWdlLCBudW1lcmFjeV90b3RhbCwgcmVhc29uaW5nX3RvdGFsLCBvcGVubWluZGVkX3RvdGFsLCB0aGlua2luZ190b3RhbCkgJT4lIAogIHNraW1yOjpza2ltKCkKYGBgCgpgYGB7ciBlY2hvPUZ9CmRhdGFfZXhwMSAlPiUgCiAgZ3JvdXBfYnkoc3ViamVjdCkgJT4lIAogIGZpbHRlcihyb3dfbnVtYmVyKCk9PTEpICU+JSAKICB1bmdyb3VwICgpICU+JSAKICBkcGx5cjo6c2VsZWN0IChhZ2UsIG51bWVyYWN5X3RvdGFsLCByZWFzb25pbmdfdG90YWwsIG9wZW5taW5kZWRfdG90YWwsIHRoaW5raW5nX3RvdGFsKSAlPiUgCiAgc2tpbXI6OnNraW0oKSAlPiUKICBrbml0cjo6a2FibGUoKSAlPiUKICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIsICJib3JkZXJlZCIsICJjb25kZW5zZWQiKSwgZml4ZWRfdGhlYWQgPSBULCBmdWxsX3dpZHRoID0gRiklPiUKICBzY3JvbGxfYm94KHdpZHRoID0gIjc4MHB4IikKYGBgCgoKIyMjIEV4ZXJjaXNlCgpGb3IgdGhpcyBleGVyY2lzZSwgd2UgdXNlIGEgZGF0YXNldCBvZiBvbmUgb2YgbXkgb3duIHN0dWRpZXMuIEluIHRoaXMgc3R1ZHksIHdlIGFza2VkIHBhcnRpY2lwYW50cyB0byBndWVzcyB0aGUgcGh5c2ljYWwgYnJpZ2h0bmVzcyBvZiByZWFzb25pbmcgYXJndW1lbnRzIGFuZCB0aGVuIHdlIGdhdmUgYSBjb2duaXRpdmUgYWJpbGl0eSB0ZXN0LiAoU2VlIHRoZSBvcmlnaW5hbCBzdHVkeSBbaGVyZV0oaHR0cHM6Ly9vc2YuaW8vZWJ4bmYvKSkuIE9wZW4gYGdoYXNlbWlfYnJpZ2h0bmVzc19leHA0LmNzdmAgZmlsZSBhbmQgYW5zd2VyIHRvIHRoZSBmb2xsb3dpbmcgcXVlc3Rpb25zOgoKMS4gSG93IG1hbnkgcGFydGljaXBhbnRzIGRpZCB3ZSB0ZXN0IGluIHRvdGFsPwoyLiBGaW5kIG91dCBob3cgbWFueSBtYWxlIGFuZCBmZW1hbGUgd2UgdGVzdGVkLgozLiBDYWxjdWxhdGUgbWVhbiBhbmQgc2QgZm9yIGFnZSBhbmQgY29nbml0aXZlIGFiaWxpdHkgKGBjb2dfYWJpbGl0eWApLgoKCmBgYHtyIHdhcm5pbmc9RiwgbWVzc2FnZT1GfQpnaGFzZW1pX2RhdGEgPC0gcmVhZF9jc3YoaGVyZSgiY2xlYW5lZF9kYXRhIiwiZ2hhc2VtaV9icmlnaHRuZXNzX2V4cDQuY3N2IikpCgpnaGFzZW1pX2RhdGEgJT4lIHN1bW1hcmlzZShuID0gbl9kaXN0aW5jdChwYXJ0aWNpcGFudCkpICMgbnVtYmVyIG9mIHBhcnRpY2lwYW50czoyMDAKCmdoYXNlbWlfZGF0YSAlPiUgZ3JvdXBfYnkgKHBhcnRpY2lwYW50KSAlPiUgZmlsdGVyIChyb3dfbnVtYmVyKCk9PTEpICU+JSBncm91cF9ieSAoZ2VuZGVyKSAlPiUgc3VtbWFyaXNlKG49IG4oKSkgJT4lIHVuZ3JvdXAoKSAjIDE4MyBmZW1hbGUsIDE3IG1hbGUKCmdoYXNlbWlfZGF0YSAlPiUgZHBseXI6OnNlbGVjdCAoYWdlLCBjb2dfYWJpbGl0eSkgJT4lIHNraW1yOjpza2ltKCkgIyBtZWFuIGFuZCBzZCBmb3IgYWdlIGFuZCBjb2duaXRpdmUgYWJpbGl0eQpgYGAKCgojIERhdGEgVmlzdWFsaXphdGlvbgoKRmlyc3QsIHdlIG5lZWQgdG8gY3JlYXRlIGEgZGF0YXNldCB3aXRoIGFnZ3JlZ2F0ZWQgYHRydXRoIGVzdGltYXRlYCBzY29yZXMgb3ZlciBgZ3JvdXBgIGFuZCBgc3RhZ2VgLiBXZSB3aWxsIHVzZSB0aGlzIGRhdGFzZXQgZm9yIGxpbmUgYW5kIGJhciBncmFwaHMuCgpgYGB7ciBtZXNzYWdlPUYsIHdhcm5pbmc9RiwgZHBpPSAzMDAsIGZpZy5oZWlnaHQ9MywgZmlnLndpZHRoPTV9CgphZ2dyZWdhdGVkX2RhdGFfZXhwMSA8LSBkYXRhX2V4cDEgJT4lCiAgZ3JvdXBfYnkoc3RhZ2UsIGdyb3VwKSAlPiUKICBtdXRhdGUodHJ1dGhfZXN0aW1hdGUgPSBtZWFuKHRydXRoX2VzdGltYXRlKSkgJT4lCiAgdW5ncm91cCgpCgpiYXJwbG90X2V4cDEgPC0gYWdncmVnYXRlZF9kYXRhX2V4cDEgJT4lCiAgZ2dwbG90KGFlcyh4PXN0YWdlLCB5PSB0cnV0aF9lc3RpbWF0ZSwgZmlsbD1ncm91cCkpICsKICBnZW9tX2JhcihzdGF0ID0gImlkZW50aXR5IiwgcG9zaXRpb249ICJkb2RnZSIpKwogICMgc3RhdF9zdW1tYXJ5KGZ1bj0gbWVhbiwgZ2VvbSA9ICJiYXIiLCBwb3NpdGlvbiA9ICJkb2RnZSIpKyAjIGNhbiBiZSB1c2VkIGluc3RlYWQgb2YgZ2VvbV9iYXIoKSBmb3IgbG9uZyBkYXRhZnJhbWVzCiAgbGFicyAoeD0gJycsIHk9ICJUcnV0aCBMaWtlbGloaG9kIEVzdGltYXRlIikgKyAKICB0aGVtZV9idygpICsgCiAgc2NhbGVfZmlsbF9qYW1hKCkgCgpiYXJwbG90X2V4cDEKCgpiYXJwbG90X2ZhY2V0X2V4cDEgPC0gYWdncmVnYXRlZF9kYXRhX2V4cDEgJT4lCiAgZ2dwbG90KGFlcyh4PWdyb3VwLCB5PSB0cnV0aF9lc3RpbWF0ZSwgZmlsbD1zdGFnZSkpICsKICBnZW9tX2JhcihzdGF0ID0gImlkZW50aXR5IiwgcG9zaXRpb249ICJkb2RnZSIpKwogIGxhYnMgKHg9ICcnLCB5PSAiVHJ1dGggTGlrZWxpaGhvZCBFc3RpbWF0ZSIpICsgCiAgdGhlbWVfYncoKSArIAogIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIiwKICAgICAgICBheGlzLnRleHQ9ZWxlbWVudF90ZXh0KHNpemU9MTEpLAogICAgICAgIGF4aXMudGl0bGUgPSBlbGVtZW50X3RleHQoc2l6ZSA9IDEyKSkgKwogIGZhY2V0X3dyYXAofnN0YWdlKSsKICBzY2FsZV9maWxsX2pjbygpIAoKYmFycGxvdF9mYWNldF9leHAxCgoKbGluZXBsb3RfZXhwMSA8LSBhZ2dyZWdhdGVkX2RhdGFfZXhwMSAlPiUKICBnZ3Bsb3QoYWVzKHg9ZmFjdG9yKHN0YWdlKSwgeT0gdHJ1dGhfZXN0aW1hdGUsIGdyb3VwPSBncm91cCwgY29sb3I9IGdyb3VwKSkgKwogIGdlb21fbGluZShhZXMobGluZXR5cGU9IGdyb3VwKSkgKwogIGdlb21fcG9pbnQoc2l6ZT0gNSkrCiAgbGFicyAoeD0gJycsIHk9ICJUcnV0aCBMaWtlbGloaG9kIEVzdGltYXRlIikgKyAKICB0aGVtZV9jbGFzc2ljKCkgKwogIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJib3R0b20iLAogICAgICAgIGF4aXMudGV4dD1lbGVtZW50X3RleHQoc2l6ZT0xMSksCiAgICAgICAgYXhpcy50aXRsZSA9IGVsZW1lbnRfdGV4dChzaXplID0gMTIpKSArCiAgc2NhbGVfY29sb3JfbmVqbSgpIAoKbGluZXBsb3RfZXhwMQoKCnZpb2xpbnBsb3RfZXhwMSA8LSBkYXRhX2V4cDEgJT4lCiAgZ2dwbG90KGFlcyh4PWZhY3RvcihzdGFnZSksIHk9IHRydXRoX2VzdGltYXRlLCBmaWxsPSBncm91cCkpICsKICBnZW9tX3Zpb2xpbigpKwogIGxhYnMgKHg9ICcnLCB5PSAiVHJ1dGggTGlrZWxpaGhvZCBFc3RpbWF0ZSIpICsgCiAgdGhlbWVfYncoKSArIAogIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJib3R0b20iLAogICAgICAgIGF4aXMudGV4dD1lbGVtZW50X3RleHQoc2l6ZT0xMSksCiAgICAgICAgYXhpcy50aXRsZSA9IGVsZW1lbnRfdGV4dChzaXplID0gMTIpKSArCiAgc2NhbGVfZmlsbF9kMygpIAoKdmlvbGlucGxvdF9leHAxCgoKYm94cGxvdF9leHAxIDwtIGRhdGFfZXhwMSAlPiUKICBnZ3Bsb3QoYWVzKHg9ZmFjdG9yKHN0YWdlKSwgeT0gdHJ1dGhfZXN0aW1hdGUsIGZpbGw9IGdyb3VwKSkgKwogIGdlb21fYm94cGxvdCgpKwogICNnZW9tX3BvaW50KHBvc2l0aW9uID0gcG9zaXRpb25fZG9kZ2Uod2lkdGg9MC43NSksIGFscGhhPSAuNSkrCiAgbGFicyAoeD0gJycsIHk9ICJUcnV0aCBMaWtlbGloaG9kIEVzdGltYXRlIikgKyAKICB0aGVtZV9idygpICsgCiAgdGhlbWUobGVnZW5kLnBvc2l0aW9uID0gImJvdHRvbSIsCiAgICAgICAgYXhpcy50ZXh0PWVsZW1lbnRfdGV4dChzaXplPTExKSwKICAgICAgICBheGlzLnRpdGxlID0gZWxlbWVudF90ZXh0KHNpemUgPSAxMikpICsKICBzY2FsZV9maWxsX3NpbXBzb25zKCkgCgpib3hwbG90X2V4cDEKCgpib3hwbG90X2ZhY2V0X2V4cDEgPC0gZGF0YV9leHAxICU+JQogIGdncGxvdChhZXMoeD1mYWN0b3Ioc3RhZ2UpLCB5PSB0cnV0aF9lc3RpbWF0ZSwgZmlsbD0gZ3JvdXApKSArCiAgZ2VvbV9ib3hwbG90KCkrCiAgbGFicyAoeD0gJycsIHk9ICJUcnV0aCBMaWtlbGloaG9kIEVzdGltYXRlIikgKyAKICB0aGVtZV9idygpICsgCiAgdGhlbWUobGVnZW5kLnBvc2l0aW9uID0gImJvdHRvbSIsCiAgICAgICAgYXhpcy50ZXh0PWVsZW1lbnRfdGV4dChzaXplPTExKSwKICAgICAgICBheGlzLnRpdGxlID0gZWxlbWVudF90ZXh0KHNpemUgPSAxMiksCiAgICAgICAgYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA5MCwgdmp1c3QgPSAwLjUsIGhqdXN0PTEpKSArCiAgZmFjZXRfd3JhcCh+Z3JvdXApKwogIHNjYWxlX2NvbG9yX3NpbXBzb25zKCkgCgpib3hwbG90X2ZhY2V0X2V4cDEKCgpgYGAKCkhvdyB0byBjb21iaW5lIG11bHRpcGxlIHBsb3RzPyBXZSBjYW4gdXNlIHRoZSBgcGF0Y2h3b3JrYCBwYWNrYWdlLiBBIG5pY2UgdHV0b3JpYWwgb24gdXNpbmcgdGhpcyBwYWNrYWdlIGNhbiBiZSBmb3VuZCBbaGVyZV0oaHR0cHM6Ly9wYXRjaHdvcmsuZGF0YS1pbWFnaW5pc3QuY29tL2FydGljbGVzL3BhdGNod29yay5odG1sKQpgYGB7ciBkcGk9IDMwMCwgZmlnLmhlaWdodD03LCBmaWcud2lkdGg9OX0KCmNvbWJpbmVkX3Bsb3RfZXhwMSA8LSAoYmFycGxvdF9mYWNldF9leHAxK2xpbmVwbG90X2V4cDEpIC8gKHZpb2xpbnBsb3RfZXhwMStib3hwbG90X2V4cDEpCmNvbWJpbmVkX3Bsb3RfZXhwMQpgYGAKCkhvdyB0byBzYXZlIGEgcGxvdD8KYGBge3JtZXNzYWdlPUZ9Cmdnc2F2ZShjb21iaW5lZF9wbG90X2V4cDEsIGZpbGVuYW1lID0gaGVyZSgib3V0cHV0cyIsImNvbWJpbmVkX3Bsb3RfZXhwMS5wbmciKSwgZHBpPTMwMCkKYGBgCgojIERhdGEgQW5hbHlzaXMKCgojIyB0LXRlc3QKCklzIHRoZXJlIGEgZGlmZmVyZW5jZSBiZXR3ZWVuIGdyb3VwcyBhdCB0aGUgZmlyc3Qgc3RhZ2U/IElkZWFsbHksIHdlIHdhbnQgcGFydGljaXBhbnRzJyByYXRpbmdzIGF0IHRoZSBmaXJzdCBzdGFnZSBiZSBzaW1pbGFyIGZvciBib3RoIGdyb3VwcyBiZWNhdXNlIHdlIGhhdmUgbm90IGRvbmUgYW55IG1hbmlwdWxhdGlvbnMuIFByZXZpb3VzIGdyYXBocyBzaG93ZWQgdXMgdGhhdCByYXRpbmdzIG9mIHNpbXBsZSBhbmQgY29tcGxleCBncm91cCBhdCB0aGlzIHN0YWdlIGFyZSBwcmV0dHkgY2xvc2UuIExldCdzIHRlc3QgdGhhdCB1c2luZyBhbiAqKmluZGVwZW5kZW50IHQtdGVzdCoqIChiZWNhdXNlIHdlIGhhdmUgMiBpbmRlcGVuZGVudCBncm91cHMpOgoKYGBge3J9CiMgSXMgdGhlcmUgYSBkaWZmZXJlbmNlIGJldHdlZW4gZ3JvdXBzIGF0IHRoZSBmaXJzdCBzdGFnZT8KZGF0YV9leHAxICU+JSAKICBncm91cF9ieShncm91cCkgJT4lIAogIGZpbHRlcihzdGFnZT09J3N0YWdlMScpICU+JSAKICB1bmdyb3VwICgpICU+JQogIHQudGVzdCh0cnV0aF9lc3RpbWF0ZX5ncm91cCwgZGF0YSA9IC4sIHBhaXJlZD1GQUxTRSkKYGBgCgpOb3csIHdlIHdvbmRlciBpZiBvcHBvc2luZyBhcmd1bWVudHMgd2VyZSBlZmZlY3RpdmUgYXQgYWxsLCByZWdhcmRsZXNzIG9mIHBhcnRpY2lwYW50cycgZ3JvdXAuIFNvLCB3ZSB3b3VsZCBsaWtlIHRvIHRlc3QgaWYgcmF0aW5ncyBhdCB0aGUgZmluYWwgc3RhZ2UgYXJlIGxvd2VyIHRoYW4gcmF0aW5ncyBhdCB0aGUgc3RhZ2UgND8gU2luY2UgYSBwYWlyIG9mIHNjb3JlIGF0IHN0YWdlIDQgYW5kIHN0YWdlIDcgaXMgY29taW5nIGZyb20gYSBzYW1lIHBlcnNvbiwgd2UgdXNlICoqcGFpcmVkIHQtdGVzdCoqLgoKYGBge3J9CiMgSXMgdGhlcmUgYSBkaWZmZXJlbmNlIGJldHdlZW4gcmF0aW5ncyBvZiBzdGFnZTQgYW5kIHN0YWdlNz8KZGF0YV9leHAxICU+JSAKICBmaWx0ZXIoc3RhZ2U9PSdzdGFnZTQnIHwgc3RhZ2U9PSdzdGFnZTcnKSAlPiUgCiAgdW5ncm91cCAoKSAlPiUKICB0LnRlc3QodHJ1dGhfZXN0aW1hdGV+c3RhZ2UsIGRhdGEgPSAuLCBwYWlyZWQ9VFJVRSkKYGBgCgoKIyMjIEV4ZXJjaXNlCgpKb2huIGV0IGFsLiAoMjAxOSkgaW52ZXN0aWdhdGVkIHRoZSBjb25zZXF1ZW5jZXMgb2YgYmFja2luZyBkb3duIChjaGFuZ2luZyBvbmUncyBtaW5kIGluIGxpZ2h0cyBvZiBldmlkZW5jZSlhbmQgaG93IG90aGVyIHBlb3BsZSB2aWV3IHNvbWVvbmUgd2hvIGNoYW5nZSB0aGVpciBtaW5kLiBJbiB0aGVpciBzZWNvbmQgZXhwZXJpbWVudHMsIHRoZXkgcHJlc2VudGVkIHBhcnRpY2lwYW50cyBlaXRoZXIgd2l0aCBhIHBlcnNvbiB3aG8gY2hhbmdlcyB0aGVpciBtaW5kIG9yIGEgcGVyc29uIHdobyByZWZ1c2VzIHRvIGJhY2sgZG93bi4gVGhlbiwgdGhleSBhc2tlZCBwYXJ0aWNpcGFudHMgdG8gcmF0ZSBob3cgaW50ZWxsaWdlbnQgYW5kIGNvbmZpZGVudCB0aGUgcGVyc29uIGlzIChTZWUgdGhlIG9yaWdpbmFsIHN0dWR5IFtoZXJlXShodHRwczovL3d3dy5oYnMuZWR1L2ZhY3VsdHkvUHVibGljYXRpb24lMjBGaWxlcy9Kb2huJTIwZXQlMjBhbCUyMC0lMjBzZWxmLXByZXNlbnRhdGlvbmFsJTIwY29uc2VxdWVuY2VzX2I4NWIyYzQzLWE1YjUtNDc0Yy05ZTJjLWU5ODUzYjEwNzI3ZS5wZGYpKS4gVGhleSByZXBvcnRlZCB0aGF0OiAKCj4gIlJlbGF0aXZlIHRvIHRoZSBlbnRyZXByZW5ldXIgd2hvIGRpZCBub3QgYmFjayBkb3duLCBwYXJ0aWNpcGFudHMganVkZ2VkIHRoZSBlbnRyZXByZW5ldXIgd2hvIGJhY2tlZCBkb3duIGFzIG1vcmUgaW50ZWxsaWdlbnQgKE1fYmFja2VkX2Rvd249NS4xMyBvdXQgb2YgNywgU0Q9MS4wOTsgTV9kaWRfbm90X2JhY2tfZG93bj0zLjk3LCBTRD0xLjU0OyB0KDI3MS4xMik94oiSNy41OSwgcCA8IC4wMDEpIGJ1dCBsZXNzIGNvbmZpZGVudCAoTV9iYWNrZWRfZG93bj00LjUwIG91dCBvZiA3LCBTRD0xLjM2OyBNX2RpZF9ub3RfYmFja19kb3duPTUuNjUsIFNEPTEuMTA7IHQoMjkxLjAxKT04LjA4LCBwIDwgLjAwMSkuIi4KCk9wZW4gdGhlIGBqb2huX2JhY2tkb3duX2V4cDIuY3N2YCBmaWxlIGFuZCB0cnkgdG8gcmVwcm9kdWNlIHRoZWlyIHJlc3VsdHMuIFJ1biB0d28gc2VwYXJhdGUgaW5kZXBlbmRlbnQgdC10ZXN0LCBvbmUgd2l0aCBgaW50ZWxsaWdlbnRgIGFzIHRoZSBkZXBlbmRlbnQgdmFyaWFibGUgYW5kIG9uZSB3aXRoIGBjb25maWRlbnRgIGFzIHRoZSBkZXBlbmRlbnQgdmFyaWFibGUuIEZvciBib3RoIHQtdGVzdCwgdXNlIGBiYWNrX2Rvd25gIGFzIHRoZSBiZXR3ZWVuLXN1YmplY3QgaW5kZXBlbmRlbnQgdmFyaWFibGUuCgpgYGB7ciBtZXNzYWdlPUYsIHdhcm5pbmc9Rn0Kam9obl9kYXRhIDwtIHJlYWRfY3N2KGhlcmUoImNsZWFuZWRfZGF0YSIsImpvaG5fYmFja2Rvd25fZXhwMi5jc3YiKSkKCgp0LnRlc3QoaW50ZWxsaWdlbnR+YmFja19kb3duLCBkYXRhID0gam9obl9kYXRhLCBwYWlyZWQ9RkFMU0UpCnQudGVzdChjb25maWRlbnR+YmFja19kb3duLCBkYXRhID0gam9obl9kYXRhLCBwYWlyZWQ9RkFMU0UpCmBgYAoKCiMjIEFuYWx5c2lzIG9mIFZhcmlhbmNlIChBTk9WQSkKCk5vdywgbGV0J3MgYW5zd2VyIG91ciBtYWluIHF1ZXN0aW9uOiBEbyBwYXJ0aWNpcGFudHMgaW4gdGhlIHNpbXBsZSBncm91cCBzaG93IGhpZ2hlciByYXRpbmdzIGZvciBzdXBwb3J0aXZlIGFyZ3VtZW50cyAoc3RhZ2UgMiB0byA0KSBhbmQgbG93ZXIgcmF0aW5ncyBmb3Igb3Bwb3NpbmcgYXJndW1lbnRzIChzdGFnZSA1IHRvIDcpLCBjb21wYXJlZCB0byBwYXJ0aWNpcGFudHMgaW4gdGhlIGNvbXBsZXggZ3JvdXA/IElmIHRoaXMgaXMgdGhlIGNhc2UuIHdlIGV4cGVjdCBhbiBpbnRlcmFjdGlvbiBpbiB0aGUgdHJhZGl0aW9uYWwgKipBbmFseXNpcyBvZiBWYXJpYW5jZSAoQU9OVkEpKiogdGVzdC4KCmBgYHtyIG1lc3NhZ2U9Riwgd2FybmluZz1GfQphb3ZfbTEgPC0gYW92X2NhciAodHJ1dGhfZXN0aW1hdGUgfiBncm91cCpzdGFnZSArCiAgICAgICAgICAgICAgICAgICAgIEVycm9yKHN1YmplY3Qvc3RhZ2UpLCBkYXRhID0gZGF0YV9leHAxKQpgYGAKCmBgYHtyIGVjaG89Rn0Ka25pdHI6OmthYmxlKG5pY2UoYW92X20xKSkgJT4lCiAga2FibGVfc3R5bGluZyhib290c3RyYXBfb3B0aW9ucyA9IGMoInN0cmlwZWQiLCAiYm9yZGVyZWQiLCAiY29uZGVuc2VkIiksIGZpeGVkX3RoZWFkID0gVCwgZnVsbF93aWR0aCA9IFQpCmBgYAoKQXMgeW91IGNhbiBzZWUsIHdlIGZvdW5kIGEgc2lnbmlmaWNhbnQgbWFpbiBlZmZlY3Qgb2Ygc3RhZ2UgYW5kIGEgc2lnbmlmaWNhbnQgZ3JvdXAgYnkgc3RhZ2UgaW50ZXJhY3Rpb24uIFdlIGNhbiB1c2UgdGhlIGBlbW1lYW5zYCBwYWNrYWdlIHRvIGRvIHBvc3QtaG9jIHRlc3RzLgoKYGBge3Igd2FybmluZz1GLCBtZXNzYWdlPUZ9CiMgbWFpbiBlZmZlY3Qgb2Ygc3RhZ2UKZW1tZWFucyhhb3ZfbTEsICdzdGFnZScpCnBhaXJzKGVtbWVhbnMoYW92X20xLCAnc3RhZ2UnKSwgYWRqdXN0PSAnaG9sbScpCmBgYAoKCmBgYHtyIHdhcm5pbmc9RiwgbWVzc2FnZT1GfQojIGdyb3VwIGJ5IHN0YWdlIGludGVyYWN0aW9uCmVtbWVhbnMoYW92X20xLCAiZ3JvdXAiLCBieT0gInN0YWdlIikKdXBkYXRlKHBhaXJzKGVtbWVhbnMoYW92X20xLCAiZ3JvdXAiLCBieT0gInN0YWdlIikpLCBieSA9IE5VTEwsIGFkanVzdCA9ICJob2xtIikgCmBgYAoKWW91IGNhbiB1c2UgdGhlIGBhZmV4X3Bsb3RgIGZ1bmN0aW9uIGZyb20gYWZleCB0byBjcmVhdGUgYmVhdXRpZnVsIHBsb3RzLiBUaG9zZSBwbG90cyBpbnRlcmFjdHMgbmljZWx5IHdpdGggZ2dwbG90OgpgYGB7ciBtZXNzYWdlPUYsIHdhcm5pbmc9RiwgZHBpPSAzMDB9CmFmZXhfcGxvdChhb3ZfbTEsIHggPSAic3RhZ2UiLCB0cmFjZSA9ICJncm91cCIsIGVycm9yPSdiZXR3ZWVuJywKICAgICAgICAgIGxpbmVfYXJnID0gbGlzdChzaXplPTEpLAogICAgICAgICAgcG9pbnRfYXJnID0gbGlzdChzaXplPTMuNSksCiAgICAgICAgICBkYXRhX2FyZyA9IGxpc3Qoc2l6ZT0gMSwgY29sb3I9ICdncmV5Jywgd2lkdGg9LjQpLAogICAgICAgICAgZGF0YV9nZW9tID0gZ2VvbV9ib3hwbG90LAogICAgICAgICAgbWFwcGluZyA9IGMoImxpbmV0eXBlIiwgInNoYXBlIiwgImZpbGwiKSwKICAgICAgICAgIGxlZ2VuZF90aXRsZSA9ICJHcm91cCIpICsKICBsYWJzKHkgPSAiVHJ1dGggTGlrZWxpaGhvZCBFc3RpbWF0ZSIsIHggPSAiIikgKwogIHRoZW1lX2J3KCkrICMgcmVtb3ZlIHRoZSBncmV5IGJhY2tncm91bmQgYW5kIGdyaWQKICB0aGVtZShheGlzLnRleHQ9ZWxlbWVudF90ZXh0KHNpemU9MTMpLAogICAgICAgIGF4aXMudGl0bGUgPSBlbGVtZW50X3RleHQoc2l6ZSA9IDEzKSwKICAgICAgICBsZWdlbmQudGV4dD1lbGVtZW50X3RleHQoc2l6ZT0xMyksCiAgICAgICAgbGVnZW5kLnRpdGxlPWVsZW1lbnRfdGV4dChzaXplPTEzKSwKICAgICAgICBsZWdlbmQucG9zaXRpb249J2JvdHRvbScsCiAgICAgICAgbGVnZW5kLmtleS5zaXplID0gdW5pdCgxLCAiY20iKSwKICAgICAgICBsZWdlbmQuYmFja2dyb3VuZCA9IGVsZW1lbnRfcmVjdChjb2xvdXIgPSAnYmxhY2snLCBmaWxsID0gJ3doaXRlJywgbGluZXR5cGU9J3NvbGlkJykpKwogIHNjYWxlX2NvbG9yX3NpbXBzb25zKCkgKwogIHNjYWxlX2ZpbGxfc2ltcHNvbnMoKQpgYGAKCgpJZiB5b3UgYXJlIGludGVyZXN0ZWQgaW4gdGhpcyB0b3BpYywgY2hlY2sgb3V0IHRoaXMgbmljZSB0dXRvcmlhbCBhYm91dCBbdXNpbmcgYWZleCB0byBydW4gQU5PVkFdKGh0dHBzOi8vY3Jhbi5yLXByb2plY3Qub3JnL3dlYi9wYWNrYWdlcy9hZmV4L3ZpZ25ldHRlcy9hZmV4X2Fub3ZhX2V4YW1wbGUuaHRtbCksIGFuZCBhbHNvIHRoaXMgaW50ZXJlc3RpbmcgdHV0b3JpYWwgb24gdGhlIFtlbW1lYW5zIHBhY2thZ2VdKGh0dHBzOi8vYW9zbWl0aC5yYmluZC5pby8yMDE5LzAzLzI1L2dldHRpbmctc3RhcnRlZC13aXRoLWVtbWVhbnMvKS4KCiMjIyBFeGVyY2lzZQoKUm90ZWxsbyBldCBhbC4gKDIwMTgpIGludmVzdGlnYXRlZCB0aGUgYXNzb2NpYXRpb24gYmV0d2VlbiB0aGUgcmFjZSAoV2hpdGUgdnMuIEJsYWNrIGZhY2VzKSBhbmQgdGhlIGd1bi10b29sIGp1ZGdtZW50cy4gSW4gdGhlaXIgZmlyc3QgZXhwZXJpbWVudHMsIHRoZXkgcHJlc2VudGVkIHBhcnRpY2lwYW50cyB3aXRoIDE2IFdoaXRlIG1hbGUgZmFjZXMgYW5kIDE2IEJsYWNrIG1hbGUgZmFjZXMsIGFuZCBmb2xsb3dpbmcgdGhhdCA4IGltYWdlcyBvZiBndW5zIGFuZCA4IGltYWdlcyBvZiB0b29scy4gVGhleSBhc2tlZCBwYXJ0aWNpcGFudHMgdG8ganVkZ2UgaWYgdGhlIG9iamVjdCBpcyBhIHRvb2wgb3IgYSBndW4gYnkgcHJlc3Npbmcga2V5Ym9hcmQgYnV0dG9ucy4gVGhlbiwgdGhleSByYW4gYW4gQU5PVkEgdG8gc2VlIGlmIHBhcnRpY2lwYW50cycgZ3VuIHJlc3BvbnNlcyBhcmUgaGlnaGVyIGZvciBhbnkgb2YgdGhlIHJhY2VzLiBTbywgdGhleSBpbmNsdWRlZCBwcmltZSByYWNlIChCbGFjaywgV2hpdGUpIGFuZCB0YXJnZXQgaWRlbnRpdHkgKGd1biwgdG9vbCkgYXMgaW5kZXBlbmRlbnQgdmFyaWFibGVzIGFuZCBwYXJ0aWNpcGFudHMnIGd1biByZXNwb25zZXMgYXMgZGVwZW5kZW50IHZhcmlhYmxlIGludG8gdGhlaXIgbGluZWFyIG1vZGVsIChTZWUgdGhlIG9yaWdpbmFsIHN0dWR5IFtoZXJlXShodHRwczovL3BzeWFyeGl2LmNvbS9hN2s5NikpLiBUaGV5IGZvdW5kIHRoYXQ6IAoKPiAiUGFydGljaXBhbnRzIG1hZGUgbW9yZSBndW4gcmVzcG9uc2VzIHRvIGd1bnMgdGhhbiB0byB0b29scywgRigxLDQ1KSA9IDUzMjQzLCBwIDwgMC4wMDAxLCDOtzJnID0gMC45OTguIEhvd2V2ZXIsIHRoZSByYWNlIG9mIHRoZSBwcmltZSBmYWNlIGRpZCBub3QgbWF0dGVyLCBGKDEsNDUpID0gMC4yODcsIHAgPiAwLjU5LCDOtzJnID0gMC4wMDEsIG5vciB3YXMgdGhlcmUgYW4gaW50ZXJhY3Rpb24gb2YgcHJpbWUgcmFjZSB3aXRoIHRhcmdldCBvYmplY3QsIEYoMSw0NSkgPSAwLjAyMiwgcCA+IDAuODgsIM63MmcgPSAwLjAwMCkiLgoKT3BlbiB0aGUgYHJvdGVsbG9fc2hvb3Rlcl9leHAxLmNzdmAgZmlsZSBhbmQgdHJ5IHRvIHJlcHJvZHVjZSB0aGVpciByZXN1bHRzLiBSdW4gYW4gQU5PVkEgKHR5cGUgSUlJKSB3aXRoIGByZXNwYCBhcyB0aGUgZGVwZW5kZW50IHZhcmlhYmxlIGFuZCB0YXJnZXQsIHByaW1lLCBhbmQgdGhlaXIgaW50ZXJhY3Rpb24gYXMgaW5kZXBlbmRlbnQgdmFyaWFibGVzLgoKCmBgYHtyIG1lc3NhZ2U9Riwgd2FybmluZz1GfQojIGxvYWQgdGhlIGdlbmVyYWwgZGF0YSBmaWxlCnJvdGVsbG9fZGF0YSA8LSByZWFkX2NzdihoZXJlKCJjbGVhbmVkX2RhdGEiLCJyb3RlbGxvX3Nob290ZXJfZXhwMS5jc3YiKSkKCiMgQU5PVkEKcm90ZWxsb19hb3YgPC0gYW92X2NhciAocmVzcCB+IHRhcmdldCpwcmltZSArCiAgICAgICAgICAgRXJyb3Ioc3ViamVjdC90YXJnZXQqcHJpbWUpLCBkYXRhID0gcm90ZWxsb19kYXRhKQpgYGAKCmBgYHtyIGVjaG89Rn0Ka25pdHI6OmthYmxlKG5pY2Uocm90ZWxsb19hb3YpKSAlPiUKICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIsICJib3JkZXJlZCIsICJjb25kZW5zZWQiKSwgZml4ZWRfdGhlYWQgPSBULCBmdWxsX3dpZHRoID0gVCkKYGBgCgoKCiMjIENvcnJlbGF0aW9uCgpOb3csIGxldCdzIGFuc3dlciB0byBhbm90aGVyIHF1ZXN0aW9uIG9mIHRoaXMgc3R1ZHk6IGRvZXMgcGVyc3Vhc2lvbiBhbmQgZGlzc3Vhc2lvbiBpcyByZWxhdGVkIHRvIG9wZW4tbWluZGVkbmVzcywgY29nbml0aXZlIGFiaWxpdHksIHJlYXNvbmluZyBhYmlsaXRpZXMsIGFuZCBjb2duaXRpdmUgc3R5bGU/IFRvIGFuc3dlciB0aGlzIHF1ZXN0aW9uLCB3ZSBuZWVkIHRvIGNyZWF0ZSB0d28gaW5kZXhlcyAoc2NvcmVzKSBvbmUgZm9yIHBlcnN1YXNpb24gYW5kIG9uZSBmb3IgZGlzc3Vhc2lvbi4gVGhlbiB3ZSBjYW4gZG8gYSBjb3JyZWxhdGlvbiB0ZXN0OgoKYGBge3IgbWVzc2FnZT1GLCBldmFsPUYsIGZpZy5hbGlnbj0nY2VudGVyJywgZHBpPTMwMH0KCmNvcl9kYXRhX2V4cDEgPC0gZGF0YV9leHAxICU+JSAKICBwaXZvdF93aWRlcihuYW1lc19mcm9tID0gc3RhZ2UsIHZhbHVlc19mcm9tID0gdHJ1dGhfZXN0aW1hdGUpICU+JQogIGdyb3VwX2J5KHN1YmplY3QpICU+JQogIG11dGF0ZShwZXJzdWFzaW9uX2luZGV4PSBzdGFnZTIrIHN0YWdlMysgc3RhZ2U0IC0gc3RhZ2UxLAogICAgICAgICBkaXNzdWFzaW9uX2luZGV4PSAoMTAxLXN0YWdlNSkgKyAoMTAxLXN0YWdlNikgKyAoMTAxLXN0YWdlNykgLSAoMTAxLXN0YWdlNCkpICU+JQogIHVuZ3JvdXAoKSU+JQogIGRwbHlyOjpzZWxlY3QocGVyc3Vhc2lvbl9pbmRleCxkaXNzdWFzaW9uX2luZGV4LG9wZW5taW5kZWRfdG90YWwsbnVtZXJhY3lfdG90YWwsdGhpbmtpbmdfdG90YWwscmVhc29uaW5nX3RvdGFsKQoKIy0tLS0tLS0tLS0gQmFzZSBSOgpjb3IoY29yX2RhdGFfZXhwMSwgbWV0aG9kID0gInBlYXJzb24iLCAgdXNlID0gImNvbXBsZXRlLm9icyIpCgojLS0tLS0tLS0tLSBQc3ljaCBsaWJyYXJ5Ogpjb3JfZGF0YV9leHAxICU+JSAKICBwc3ljaDo6cGFpcnMucGFuZWxzKG1ldGhvZCA9ICJwZWFyc29uIiwgaGlzdC5jb2wgPSAiIzAwQUZCQiIsIGRlbnNpdHkgPSBULCBlbGxpcHNlcyA9IEYsIHN0YXJzID0gVCkKCiMtLS0tLS0tLS0tIENvcnJlbGF0aW9uIGxpYnJhcnk6CmNvcnJlbGF0aW9uOjpjb3JyZWxhdGlvbihjb3JfZGF0YV9leHAxKSAlPiUgc3VtbWFyeSgpCgojLS0tLS0tLS0tLSBhcGFUYWJsZXMgbGlicmFyeToKY29yX2RhdGFfZXhwMSAlPiUgCiAgYXBhVGFibGVzOjphcGEuY29yLnRhYmxlKGZpbGVuYW1lPSIuL291dHB1dHMvQ29yTWF0cml4LmRvYyIsIHNob3cuY29uZi5pbnRlcnZhbD1UKQpgYGAKCmBgYHtyIG1lc3NhZ2U9RiwgZWNobz1GLCBmaWcuYWxpZ249J2NlbnRlcicsIGRwaT0zMDB9CmNvcl9kYXRhX2V4cDEgPC0gZGF0YV9leHAxICU+JSAKICBwaXZvdF93aWRlcihuYW1lc19mcm9tID0gc3RhZ2UsIHZhbHVlc19mcm9tID0gdHJ1dGhfZXN0aW1hdGUpICU+JQogIGdyb3VwX2J5KHN1YmplY3QpICU+JQogIG11dGF0ZShwZXJzdWFzaW9uX2luZGV4PSBzdGFnZTIrIHN0YWdlMysgc3RhZ2U0IC0gc3RhZ2UxLAogICAgICAgICBkaXNzdWFzaW9uX2luZGV4PSAoMTAxLXN0YWdlNSkgKyAoMTAxLXN0YWdlNikgKyAoMTAxLXN0YWdlNykgLSAoMTAxLXN0YWdlNCkpICU+JQogIHVuZ3JvdXAoKSU+JQogIGRwbHlyOjpzZWxlY3QocGVyc3Vhc2lvbl9pbmRleCxkaXNzdWFzaW9uX2luZGV4LG9wZW5taW5kZWRfdG90YWwsbnVtZXJhY3lfdG90YWwsdGhpbmtpbmdfdG90YWwscmVhc29uaW5nX3RvdGFsKQoKIy0tLS0tLS0tLS0gQmFzZSBSOgpjb3IoY29yX2RhdGFfZXhwMSwgbWV0aG9kID0gInBlYXJzb24iLCAgdXNlID0gImNvbXBsZXRlLm9icyIpJT4lCiAga25pdHI6OmthYmxlKGRpZ2l0cyA9IDIpICU+JQogIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIiwgImJvcmRlcmVkIiwgImNvbmRlbnNlZCIpLCBmaXhlZF90aGVhZCA9IFQsIGZ1bGxfd2lkdGggPSBGKSU+JQogIHNjcm9sbF9ib3god2lkdGggPSAiNzgwcHgiKQoKIy0tLS0tLS0tLS0gUHN5Y2ggbGlicmFyeToKY29yX2RhdGFfZXhwMSAlPiUgCiAgcHN5Y2g6OnBhaXJzLnBhbmVscyhtZXRob2QgPSAicGVhcnNvbiIsIGhpc3QuY29sID0gIiMwMEFGQkIiLCBkZW5zaXR5ID0gVCwgZWxsaXBzZXMgPSBGLCBzdGFycyA9IFQpCgojLS0tLS0tLS0tLSBDb3JyZWxhdGlvbiBsaWJyYXJ5Ogpjb3JyZWxhdGlvbjo6Y29ycmVsYXRpb24oY29yX2RhdGFfZXhwMSkgJT4lIHN1bW1hcnkoKSU+JQogIGtuaXRyOjprYWJsZShkaWdpdHMgPSAyKSAlPiUKICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIsICJib3JkZXJlZCIsICJjb25kZW5zZWQiKSwgZml4ZWRfdGhlYWQgPSBULCBmdWxsX3dpZHRoID0gRiklPiUKICBzY3JvbGxfYm94KHdpZHRoID0gIjc4MHB4IikKCmBgYAoKCgojIyMgRXhlcmNpc2UKClBlbm55Y29vayBldCBhbC4gKDIwMjApIGludmVzdGlnYXRlZCB0aGUgcmVsYXRpb25zaGlwIGJldHdlZW4gYWN0aXZlbHkgb3Blbi1taW5kZWQgdGhpbmtpbmcgc3R5bGUgYWJvdXQgZXZpZGVuY2UgKEFPVC1FKSBhbmQgZGlmZmVyZW50IHBvbGl0aWNhbCwgc2NpZW50aWZpYywgYW5kIHJlbGlnaW91cyBiZWxpZWZzIChzZWUgdGhlIG9yaWdpbmFsIHBhcGVyIFtoZXJlXShodHRwczovL3BzeWFyeGl2LmNvbS9hN2s5NikpLiBJbiB0aGVpciBmaXJzdCBleHBlcmltZW50LCB0aGV5IGNhbGN1bGF0ZWQgdGhlIGNvcnJlbGF0aW9uIG9mIEFPVEUgYW5kIHNjaWVudGlmaWMgYmVsaWVmcyBpdGVtcyAoZ2xvYmFsIHdhcm1pbmcsIGV2b2x1dGlvbiwgZXRjLikgYW5kIHRoZXkgZm91bmQgdGhlIGZvbGxvd2luZyByZXN1bHRzOgoKYGBge3IgZWNobz1GQUxTRSwgb3V0LndpZHRoPSI3MDBweCIsIG91dC5oZWlnaHQ9IjM1MHB4IiwgZmlnLmNhcD0gImFkYXB0ZWQgZnJvbSBbUGVubnljb29rIGV0IGFsLiAoMjAyMCldKGh0dHBzOi8vcHN5YXJ4aXYuY29tL2E3azk2KSJ9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKGhlcmUoJ2lucHV0cycsJ3Blbm55Y29va19jb3JyLnBuZycpKQpgYGAKCk9wZW4gdGhlIGBwZW5ueWNvb2tfYW90ZV9leHAxLmNzdmAgZmlsZSBhbmQgdHJ5IHRvIHJlcHJvZHVjZSB0aGVpciByZXN1bHRzIGJ5IGNyZWF0aW5nIHRoZSBzYW1lIGNvcnJlbGF0aW9uIG1hdHJpeC4KCmBgYHtyIG1lc3NhZ2U9RiwgZXZhbD1GfQpwZW5ueWNvb2tfZGF0YSA8LSByZWFkX2NzdihoZXJlKCJjbGVhbmVkX2RhdGEiLCJwZW5ueWNvb2tfYW90ZV9leHAxLmNzdiIpKSAKCgojLS0tLS0tLS0tLSBCYXNlIFI6CmNvcihwZW5ueWNvb2tfZGF0YSwgbWV0aG9kID0gInBlYXJzb24iLCAgdXNlID0gImNvbXBsZXRlLm9icyIpCgojLS0tLS0tLS0tLSBQc3ljaCBsaWJyYXJ5OgpwZW5ueWNvb2tfZGF0YSAlPiUgCiAgcHN5Y2g6OnBhaXJzLnBhbmVscyhtZXRob2QgPSAicGVhcnNvbiIsIGhpc3QuY29sID0gIiMwMEFGQkIiLCBkZW5zaXR5ID0gVCwgZWxsaXBzZXMgPSBGLCBzdGFycyA9IFQpCgojLS0tLS0tLS0tLSBDb3JyZWxhdGlvbiBsaWJyYXJ5Ogpjb3JyZWxhdGlvbjo6Y29ycmVsYXRpb24ocGVubnljb29rX2RhdGEpICU+JSBzdW1tYXJ5KCkKCiMtLS0tLS0tLS0tIGFwYVRhYmxlcyBsaWJyYXJ5OgpwZW5ueWNvb2tfZGF0YSAlPiUgCiAgYXBhVGFibGVzOjphcGEuY29yLnRhYmxlKGZpbGVuYW1lPSIuL291dHB1dHMvQ29yTWF0cml4LmRvYyIsIHNob3cuY29uZi5pbnRlcnZhbD1UKQpgYGAKCgpgYGB7ciBtZXNzYWdlPUYsIGV2YWw9VCwgZWNobz1GLCBmaWcuYWxpZ249J2NlbnRlcicsIGRwaT0zMDB9CnBlbm55Y29va19kYXRhIDwtIHJlYWRfY3N2KGhlcmUoImNsZWFuZWRfZGF0YSIsInBlbm55Y29va19hb3RlX2V4cDEuY3N2IikpICU+JQogIGNsZWFuX25hbWVzKCkKCmNvcnJlbGF0aW9uOjpjb3JyZWxhdGlvbihwZW5ueWNvb2tfZGF0YSkgJT4lIHN1bW1hcnkoKSAlPiUKICBrbml0cjo6a2FibGUoZGlnaXRzID0gMikgJT4lCiAga2FibGVfc3R5bGluZyhib290c3RyYXBfb3B0aW9ucyA9IGMoInN0cmlwZWQiLCAiYm9yZGVyZWQiLCAiY29uZGVuc2VkIiksIGZpeGVkX3RoZWFkID0gVCwgZnVsbF93aWR0aCA9IEYpJT4lCiAgc2Nyb2xsX2JveCh3aWR0aCA9ICI3ODBweCIpCgpgYGAKCgojIyBMaW5lYXIgUmVncmVzc2lvbgoKSW4gdGhlIHByZXZpb3VzIHNlY3Rpb24sIHdlIGZvdW5kIHRoYXQgb3Blbi1taW5kZWRuZXNzIChBT1QtRSkgaXMgY29ycmVsYXRlZCB3aXRoIHBlcnN1YXNpb24uIE5vdywgb25lIG1heSBhc2sgaWYgb3Blbi1taW5kZWRuZXNzIGNhbiBwcmVkaWN0IHBlcnN1YXNpb24gYWZ0ZXIgY29udHJvbGxpbmcgZm9yIHJlYXNvbmluZyBhbmQgY29udHJvbGxpbmcgYWJpbGl0aWVzPyBUbyBhbnN3ZXIgdGhhdCwgd2UgY2FuIHJ1biBhIG11bHRpcGxlIHJlZ3Jlc3Npb24gYW5hbHlzaXM6CmBgYHtyfQpleHAxX3JlZz1sbShwZXJzdWFzaW9uX2luZGV4IH4gb3Blbm1pbmRlZF90b3RhbCsgbnVtZXJhY3lfdG90YWwrIHRoaW5raW5nX3RvdGFsKyByZWFzb25pbmdfdG90YWwsCiAgICAgICAgICAgICAgICAgIGRhdGE9Y29yX2RhdGFfZXhwMSkKYGBgCgpgYGB7ciBtZXNzYWdlPUYsIGV2YWw9VCwgZWNobz1GLCBmaWcuYWxpZ249J2NlbnRlcicsIGRwaT0zMDB9CmJyb29tOjp0aWR5KGV4cDFfcmVnKSU+JQogIGtuaXRyOjprYWJsZShkaWdpdHMgPSAyKSAlPiUKICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIsICJib3JkZXJlZCIsICJjb25kZW5zZWQiKSwgZml4ZWRfdGhlYWQgPSBULCBmdWxsX3dpZHRoID0gVCkKYGBgCgojIyMgRXhlcmNpc2UKClRyw6ltb2xpw6hyZSBhbmQgRGplcmlvdWF0ICgyMDIwKSBleGFtaW5lZCB0aGUgcm9sZSBvZiAqY29nbml0aXZlIHJlZmxlY3Rpb24qIGFuZCAqYmVsaWVmIGluIHNjaWVuY2UqIGluIGNsaW1hdGUgY2hhbmdlIHNrZXB0aWNpc20uIEluIHRoZWlyIGZpcnN0IHN0dWR5LCB0aGV5IHJldmVhbGVkIHRoYXQgY29nbml0aXZlIHJlZmxlY3Rpb24gYW5kIGJlbGllZiBpbiBzY2llbmNlIG5lZ2V0aXZlbHkgcHJlZGljdGVkIGNsaW1hdGUgY2hhbmdlIHNrZXB0aWNpc20gZXZlbiBhZnRlciBjb250cm9sbGluZyBmb3IgZGVtb2dyYXBoaWMgYW5kIGNvZ25pdGl2ZSBhYmlsaXR5IHZhcmlhYmxlcyAoc2VlIHRoZSBvcmlnaW5hbCBwYXBlciBbaGVyZV0oaHR0cHM6Ly9wc3lhcnhpdi5jb20vdnA4azYvKSkuIAoKYGBge3IgZWNobz1GQUxTRSwgb3V0LndpZHRoPSI3MDBweCIsIG91dC5oZWlnaHQ9IjM1MHB4IiwgZmlnLmNhcD0gImFkYXB0ZWQgZnJvbSBbVHLDqW1vbGnDqHJlIGFuZCBEamVyaW91YXQgKDIwMjApXShodHRwczovL3BzeWFyeGl2LmNvbS92cDhrNi8pIn0Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoaGVyZSgnaW5wdXRzJywndHJlbW9saWVyZV9yZWcucG5nJykpCmBgYAoKT3BlbiB0aGUgYHRyZW1vbGllcmVfZGF0YV9leHAxLmNzdmAgZmlsZSBhbmQgdHJ5IHRvIHJlcHJvZHVjZSB0aGVpciByZXN1bHRzIGJ5IHJ1bm5pbmcgYSBtdWx0aXBsZSBsaW5lYXIgcmVncmVzc2lvbi4gRW50ZXIgYWdlLCBnZW5kZXIsIGVkdWNhdGlvbiwgYmVsaWVmIGluIHNjaWVuY2UsIGxpdGVyYWN5LCBudW1lcmFjeSAoTnVtdG90YWwpLCBhbmQgY29nbml0aXZlIHJlZmxlY3Rpb24gYXMgcHJlZGljdG9ycyBhbmQgZW50ZXIgY2xpbWF0ZSBjaGFuZ2Ugc2tlcHRpY2lzbSAoY2xpbWF0bykgYXMgdGhlIG91dGNvbWUgdmFyaWFibGUuCgpgYGB7ciBtZXNzYWdlPUZ9ClRyZW1vbGllcmVfZGF0YSA8LSByZWFkX2NzdihoZXJlKCJjbGVhbmVkX2RhdGEiLCJ0cmVtb2xpZXJlX2RhdGFfZXhwMS5jc3YiKSkKClRyZW1vbGllcmVfcmVnPWxtKENsaW1hdG8gfiBBZ2UrIEdlbmRlcisgRWR1Y2F0aW9uKyBCZWxpZWZJblNjaWVuY2V0b3RhbCsgTGl0ZXJhY3krIE51bXRvdGFsKyBDb2duaXRpdmVSZWZsZWN0aW9uLAogICAgICAgICAgICAgICAgICAgIGRhdGE9VHJlbW9saWVyZV9kYXRhKQpgYGAKCgpgYGB7ciBtZXNzYWdlPUYsIGV2YWw9VCwgZWNobz1GLCBmaWcuYWxpZ249J2NlbnRlcicsIGRwaT0zMDB9CmJyb29tOjp0aWR5KFRyZW1vbGllcmVfcmVnKSU+JQogIGtuaXRyOjprYWJsZShkaWdpdHMgPSAyKSAlPiUKICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIsICJib3JkZXJlZCIsICJjb25kZW5zZWQiKSwgZml4ZWRfdGhlYWQgPSBULCBmdWxsX3dpZHRoID0gVCkKCmdsYW5jZShUcmVtb2xpZXJlX3JlZyklPiUKICBrbml0cjo6a2FibGUoZGlnaXRzID0gMikgJT4lCiAga2FibGVfc3R5bGluZyhib290c3RyYXBfb3B0aW9ucyA9IGMoInN0cmlwZWQiLCAiYm9yZGVyZWQiLCAiY29uZGVuc2VkIiksIGZpeGVkX3RoZWFkID0gVCwgZnVsbF93aWR0aCA9IEYpJT4lCiAgc2Nyb2xsX2JveCh3aWR0aCA9ICI3ODBweCIpCmBgYAoKCiMgUm1hcmtkb3duCgpUbyBiZSBjb21wbGV0ZWQuLi4KCgojIFJlZmVyZW5jZXMKCi0gR2hhc2VtaSwgTy4sIEhhbmRsZXksIFMuLCAmIEhvd2FydGgsIFMuICgyMDIwKS4gVGhlIEJyaWdodCBIb211bmN1bHVzIGluIG91ciBIZWFkOiBJbmRpdmlkdWFsIERpZmZlcmVuY2VzIGluIEludHVpdGl2ZSBTZW5zaXRpdml0eSB0byBMb2dpY2FsIFZhbGlkaXR5LgoKLSBKb2huLCBMLiBLLiwgSmVvbmcsIE0uLCBHaW5vLCBGLiwgJiBIdWFuZywgTC4gKDIwMTkpLiBUaGUgc2VsZi1wcmVzZW50YXRpb25hbCBjb25zZXF1ZW5jZXMgb2YgdXBob2xkaW5nIG9uZeKAmXMgc3RhbmNlIGluIHNwaXRlIG9mIHRoZSBldmlkZW5jZS4gT3JnYW5pemF0aW9uYWwgQmVoYXZpb3IgYW5kIEh1bWFuIERlY2lzaW9uIFByb2Nlc3NlcywgMTU0LCAxLTE0LgoKLSBQZW5ueWNvb2ssIEcuLCBDaGV5bmUsIEouIEEuLCBLb2VobGVyLCBELiBKLiwgJiBGdWdlbHNhbmcsIEouIEEuICgyMDIwKS4gT24gdGhlIGJlbGllZiB0aGF0IGJlbGllZnMgc2hvdWxkIGNoYW5nZSBhY2NvcmRpbmcgdG8gZXZpZGVuY2U6IEltcGxpY2F0aW9ucyBmb3IgY29uc3BpcmF0b3JpYWwsIG1vcmFsLCBwYXJhbm9ybWFsLCBwb2xpdGljYWwsIHJlbGlnaW91cywgYW5kIHNjaWVuY2UgYmVsaWVmcy4gSnVkZ21lbnQgYW5kIERlY2lzaW9uIE1ha2luZywgMTUoNCksIDQ3Ni4KCi0gUm90ZWxsbywgQy4gTS4sIEtlbGx5LCBMLiBKLiwgSGVpdCwgRS4sIFZhemlyZSwgUy4sICYgVnVsLCBFLiAoMjAxOCkuIFRoZSBTaGFwZSBvZiBST0MgQ3VydmVzIGluIFNob290ZXIgVGFza3M6IEltcGxpY2F0aW9ucyBmb3IgQmVzdCBQcmFjdGljZXMgaW4gQW5hbHlzaXMuIENvbGxhYnJhOiBQc3ljaG9sb2d5LCA0KDEpLgoKLSBUcsOpbW9sacOocmUsIEIuLCAmIERqZXJpb3VhdCwgSC4gKDIwMjApLiBEb27igJl0IHlvdSBzZWUgdGhhdCBpdHMgY29sZCEgRXhwbG9yaW5nIHRoZSByb2xlcyBvZiBjb2duaXRpdmUgcmVmbGVjdGlvbiwgY2xpbWF0ZSBzY2llbmNlIGxpdGVyYWN5LCBpbGx1c2lvbiBvZiBrbm93bGVkZ2UsIGFuZCBwb2xpdGljYWwgb3JpZW50YXRpb24gaW4gY2xpbWF0ZSBjaGFuZ2Ugc2tlcHRpY2lzbS4KCi0gV2lja2hhbSwgSC4gKDIwMTQpLiBUaWR5IGRhdGEuIEpvdXJuYWwgb2YgU3RhdGlzdGljYWwgU29mdHdhcmUsIDU5KDEwKSwgMS0yMy4=